A recommendation engine is a system that predicts the preference or rating that a user would give to an item. The system selects relevant items for the users based on their behavior and the knowledge it has about them. Nowadays, recommendation systems are used in many online services. They help users by selecting the stuff they might be interested in from the vast amount of available data irrelevant to them. Offering good recommendations enhances user engagement. E-commerce sites also benefit from offering relevant product recommendations by increasing their average sale.
We are going to create a simple yet powerful recommendation engine that suggests products that are usually bought together. We will suggest products based on historic sales, thus identifying products that are usually bought together. We are going to suggest complementary products in two different scenarios:
We are going to use Redis to store products that are purchased together. Remember that you already used Redis in Chapter 6, Tracking User Actions. If you haven't installed Redis yet, you can find installation instructions in that chapter.
Now, we will recommend products to users based on what they have added to the cart. We are going to store a key in Redis for each product bought in our site. The product key will contain a Redis sorted set with scores. We will increment the score by 1
for each product bought together every time a new purchase is completed.
When an order is successfully paid for, we store a key for each product bought, including a sorted set of products that belong to the same order. The sorted set allows us to give scores for products that are bought together.
Edit the settings.py
file of your project and add the following settings to it:
REDIS_HOST = 'localhost' REDIS_PORT = 6379 REDIS_DB = 1
These are the settings required to establish a connection with the Redis server. Create a new file inside the shop
application directory and name it recommender.py
. Add the following code to it:
import redis from django.conf import settings from .models import Product # connect to redis r = redis.StrictRedis(host=settings.REDIS_HOST, port=settings.REDIS_PORT, db=settings.REDIS_DB) class Recommender(object): def get_product_key(self, id): return 'product:{}:purchased_with'.format(id) def products_bought(self, products): product_ids = [p.id for p in products] for product_id in product_ids: for with_id in product_ids: # get the other products bought with each product if product_id != with_id: # increment score for product purchased together r.zincrby(self.get_product_key(product_id), with_id, amount=1)
This is the Recommender
class that will allow us to store product purchases and retrieve product suggestions for a given product or products. The get_product_key()
method receives an id
of a Product
object and builds the Redis key for the sorted set where related products are stored, which looks like product:[id]:purchased_with
.
The products_bought()
method receives a list of Product
objects that have been bought together (that is, belong to the same order). In this method, we perform the following tasks:
Product
objects.id
, we iterate over the product IDs and skip the same product so that we get the products that are bought together with each product.get_product_id()
method. For a product with an ID of 33
, this method returns the key product:33:purchased_with
. This is the key for the sorted set that contains the product IDs of products that were bought together with this one.id
contained in the sorted set by 1
. The score represents the times another product has been bought together with the given product.So we have a method to store and score the products that were bought together. Now we need a method to retrieve the products that are bought together for a list of given products. Add the following suggest_products_for()
method to the Recommender
class:
def suggest_products_for(self, products, max_results=6): product_ids = [p.id for p in products] if len(products) == 1: # only 1 product suggestions = r.zrange( self.get_product_key(product_ids[0]), 0, -1, desc=True)[:max_results] else: # generate a temporary key flat_ids = ''.join([str(id) for id in product_ids]) tmp_key = 'tmp_{}'.format(flat_ids) # multiple products, combine scores of all products # store the resulting sorted set in a temporary key keys = [self.get_product_key(id) for id in product_ids] r.zunionstore(tmp_key, keys) # remove ids for the products the recommendation is for r.zrem(tmp_key, *product_ids) # get the product ids by their score, descendant sort suggestions = r.zrange(tmp_key, 0, -1, desc=True)[:max_results] # remove the temporary key r.delete(tmp_key) suggested_products_ids = [int(id) for id in suggestions] # get suggested products and sort by order of appearance suggested_products = list(Product.objects.filter(id__in=suggested_products_ids)) suggested_products.sort(key=lambda x: suggested_products_ids.index(x.id)) return suggested_products
The suggest_products_for()
method receives the following parameters:
products
: This is a list of Product
objects to get recommendations for. It can contain one or more products.max_results
: This is an integer that represents the maximum number of recommendations to return.In this method, we perform the following actions:
Product
objects.ZRANGE
command. We limit the number of results to the number specified in the max_results
attribute (6
by default).ZUNIONSTORE
command. The ZUNIONSTORE
command performs a union of the sorted sets with the given keys, and stores the aggregated sum of scores of the elements in a new Redis key. You can read more about this command at http://redis.io/commands/ZUNIONSTORE. We save the aggregated scores in the temporary key.ZREM
command.ZRANGE
command. We limit the number of results to the number specified in the max_results
attribute. Then we remove the temporary key.Product
objects with the given id
and we order the products by the same order as the them.For practical purposes, let's also add a method to clear the recommendations. Add the following method to the Recommender
class:
def clear_purchases(self): for id in Product.objects.values_list('id', flat=True): r.delete(self.get_product_key(id))
Let's try our recommendation engine. Make sure you include several Product
objects in the database and initialize the Redis server using the following command from the shell:
src/redis-server
Open another shell, execute python manage.py shell
, and write the following code to retrieve several products:
from shop.models import Product black_tea = Product.objects.get(translations__name='Black tea') red_tea = Product.objects.get(translations__name='Red tea') green_tea = Product.objects.get(translations__name='Green tea') tea_powder = Product.objects.get(translations__name='Tea powder')
Then, add some test purchases to the recommendation engine:
from shop.recommender import Recommender r = Recommender() r.products_bought([black_tea, red_tea]) r.products_bought([black_tea, green_tea]) r.products_bought([red_tea, black_tea, tea_powder]) r.products_bought([green_tea, tea_powder]) r.products_bought([black_tea, tea_powder]) r.products_bought([red_tea, green_tea])
We have stored the following scores:
black_tea: red_tea (2), tea_powder (2), green_tea (1) red_tea: black_tea (2), tea_powder (1), green_tea (1) green_tea: black_tea (1), tea_powder (1), red_tea(1) tea_powder: black_tea (2), red_tea (1), green_tea (1)
Let's take a look at the recommended products for a single product:
>>> r.suggest_products_for([black_tea]) [<Product: Tea powder>, <Product: Red tea>, <Product: Green tea>] >>> r.suggest_products_for([red_tea]) [<Product: Black tea>, <Product: Tea powder>, <Product: Green tea>] >>> r.suggest_products_for([green_tea]) [<Product: Black tea>, <Product: Tea powder>, <Product: Red tea>] >>> r.suggest_products_for([tea_powder]) [<Product: Black tea>, <Product: Red tea>, <Product: Green tea>]
As you can see, the order for recommended products is based on their score. Let's get recommendations for multiple products with aggregated scores:
>>> r.suggest_products_for([black_tea, red_tea]) [<Product: Tea powder>, <Product: Green tea>] >>> r.suggest_products_for([green_tea, red_tea]) [<Product: Black tea>, <Product: Tea powder>] >>> r.suggest_products_for([tea_powder, black_tea]) [<Product: Red tea>, <Product: Green tea>]
You can see that the order of the suggested products matches the aggregated scores. For example, products suggested for black_tea
and red_tea
are tea_powder
(2+1
) and green_tea
(1+1
).
We have verified that our recommendation algorithm works as expected. Let's display recommendations for products on our site.
Edit the views.py
file of the shop
application and add the following import:
from .recommender import Recommender
Add the following code to the product_detail
view just before the render()
function:
r = Recommender()recommended_products = r.suggest_products_for([product], 4)
We get a maximum of four product suggestions. The product_detail
view should now look as follows:
from .recommender import Recommender def product_detail(request, id, slug): product = get_object_or_404(Product, id=id, slug=slug, available=True) cart_product_form = CartAddProductForm() r = Recommender() recommended_products = r.suggest_products_for([product], 4) return render(request, 'shop/product/detail.html', {'product': product, 'cart_product_form': cart_product_form, 'recommended_products': recommended_products})
Now edit the shop/product/detail.html
template of the shop
application and add the following code after {{ product.description|linebreaks }}
:
{% if recommended_products %} <div class="recommendations"> <h3>{% trans "People who bought this also bought" %}</h3> {% for p in recommended_products %} <div class="item"> <a href="{{ p.get_absolute_url }}"> <img src="{% if p.image %}{{ p.image.url }}{% else %}{% static "img/no_image.png" %}{% endif %}"> </a> <p><a href="{{ p.get_absolute_url }}">{{ p.name }}</a></p> </div> {% endfor %} </div> {% endif %}
Run the development server with the command python manage.py runserver
and open http://127.0.0.1:8000/en/
in your browser. Click on any product to see its details page. You should see that recommended products are displayed below the product, as shown in the following image:
We are also going to include product recommendations in the cart. The recommendation will be based on the products that the user has added to the cart. Edit the views.py
inside the cart
application directory and add the following import:
from shop.recommender import Recommender
Then, edit the cart_detail
view to make it look as follows:
def cart_detail(request): cart = Cart(request) for item in cart: item['update_quantity_form'] = CartAddProductForm( initial={'quantity': item['quantity'], 'update': True}) coupon_apply_form = CouponApplyForm() r = Recommender() cart_products = [item['product'] for item in cart] recommended_products = r.suggest_products_for(cart_products, max_results=4) return render(request, 'cart/detail.html', {'cart': cart, 'coupon_apply_form': coupon_apply_form, 'recommended_products': recommended_products})
Edit the cart/detail.html
template of the cart
application and add the following code just after the </table>
HTML tag:
{% if recommended_products %} <div class="recommendations cart"> <h3>{% trans "People who bought this also bought" %}</h3> {% for p in recommended_products %} <div class="item"> <a href="{{ p.get_absolute_url }}"> <img src="{% if p.image %}{{ p.image.url }}{% else %}{% static "img/no_image.png" %}{% endif %}"> </a> <p><a href="{{ p.get_absolute_url }}">{{ p.name }}</a></p> </div> {% endfor %} </div> {% endif %}
Open http://127.0.0.1:8000/en/
in your browser and add a couple of products to your cart. When you navigate to http://127.0.0.1:8000/en/cart/
, you should see the aggregated product recommendations for the items in the cart as follows:
Congratulations! You have built a complete recommendation engine using Django and Redis.