Factorization models are very popular in recommendation systems because they can be used to discover latent features underlying the interactions between two different kinds of entities. In this chapter, we will provide several examples of how to develop recommendation system for predictive analytics.
We will see the theoretical background of recommendation systems, such as matrix factorization. Later in the chapter, we will see how to use a collaborative approach to develop a movie recommendation system. Finally, will see how to use Factorization Machines (FMs) and improved versions of them to develop more robust recommendation systems.
In summary, the following topics will be covered in this chapter:
- Recommendation systems
- A movie recommendation system using the collaborative filtering approach
- K-means for clustering similar movies
- FM-based recommendation systems
- Using improved FMs for movie recommendation
Recommender techniques are nothing but information agents that try to predict items that users may be interested in and recommend the best ones to the target user. These techniques can be classified based on the information sources they use. For example, user features (age, gender, income, and location), item features (keywords and genres), user-item ratings (explicit ratings and transaction data), and other information about the user and item that are useful for the process of recommendation.
Thus, a recommendation system (otherwise known as a recommendation engine or RE) is a subclass of information filtering systems that help to predict the rating or preference, based on the rating provided by users for an item. In recent years, recommendation systems have become increasingly popular.
For example, at Amazon, the importance of suggesting the right item to the right user can be gauged by the fact that 35% of all sales are estimated to be generated by the recommendation engine. Therefore, Amazon is investing a large amount of talent and resources on getting better at AI – specifically "deep learning" technology – to make recommendation engines which learn and scale even more efficiently.
Consequently, they are being used in many areas, such as movies, music, news, books, research articles, search queries, social tags, products, jokes, restaurants, garments, financial services, life insurance, and online dating sites.
There are a couple of ways to develop REs to produce a list of recommendations. For example, collaborative and content-based filtering, knowledge-based, or the personality-based approach.
Using collaborative filtering approaches, an RE can be built based on a user's past behavior. Numerical ratings are given on consumed items. Sometimes, it can be based on the decisions made by other users who also have purchased the same items, using some widely-used data mining algorithms such as Apriori or FP-growth. In the following diagram, you can get some idea of different recommendation systems:
Figure 1: A comparative view of different recommendation systems
Collaborative filtering-based approaches often suffer from the following three problems:
- Cold start: Sometimes they can be stuck when a large amount of data about users is required to make a more accurate recommendation system.
- Scalability: A large amount of computation power is often necessary to calculate recommendations using a dataset with millions of users and products.
- Sparsity: This often happens with crowdsourced datasets when a huge number of items are sold on major e-commerce sites. All recommendation datasets are crowd-sourced in some sense. This is a general problem for almost all recommendation systems that have a sufficiently large number of items to offer to a sufficiently large number of users and need not be confined to e-commerce sites only.
In this case, active users may rate only a small subset of the whole items sold, so even the most popular items have very few ratings. Accordingly, the user versus items matrix becomes very sparse. In other words, handling a large-scale sparse matrix is computationally very challenging.
To overcome these issues, a particular type of collaborative filtering algorithm uses matrix factorization, which is a low-rank matrix approximation technique. We will see an example of this later in the chapter.
With content-based filtering approaches, a series of discrete characteristics of an item is utilized to recommend additional items with similar properties. Sometimes it is based on a description of the item and a profile of the user's preferences. These approaches try to recommend items that are similar to those that a user liked in the past, or that are being used currently.
A key issue with content-based filtering is whether the system is able to learn user preferences from their actions regarding one content source and use them with other content types. When this type of RE is deployed, it can then be used to predict items, or ratings for items, that the user may have an interest in.
As you have seen, there are several pros and cons of using collaborative filtering and content-based filtering approaches. Therefore, to overcome the limitations of these two approaches, the recent trend has shown that a hybrid approach can be more effective and accurate. Sometimes, factorization approaches such as Factorization Matrix (FM) and Singular Value Decomposition (SVD) are used to make them robust. Hybrid approaches can be implemented in several ways:
- Content-based and collaborative predictions are computed separately and later are combined into one model. In this approach, FM and SVD are often used extensively.
- Content-based capabilities are added to a collaborative approach or vice versa. Again, FM and SVD are used for better prediction.
Netflix is a perfect example of using this hybrid approach to make recommendations to their subscribers. This site makes recommendations in two ways:
- Collaborative filtering: By comparing the watching and searching habits of similar users
- Content-based filtering: By recommending movies that share characteristics with films that a user has rated highly
Collaborative filtering methods are classified as memory-based i.e. user-based algorithm and model-based collaborative filtering (kernel mapping recommended). In the model-based collaborative filtering technique, users and products are described by a small set of factors, also called latent factors (LFs).
The LFs are then used to predict the missing entries. The Alternating Least Squares (ALS) algorithm is used to learn these latent factors. There are several advantages:
- Compared to a memory-based approach, a model-based approach can handle the sparsity of the original matrix better
- Using this approach, the resulting models become much smaller than the actual dataset, which imparts scalability to the overall system.
- Model-based systems are faster than memory-based systems because the resulting model is much smaller than what you need to query the whole dataset.
- Using this approach, it is relatively easy to avoid overfitting.
As a downside, the model-based approach is not flexible and adaptable because it is difficult to add data to the model. The quality of predictions depends on the way the model is built, but since this approach is inflexible, we cannot utilize all the data. This implies that we may not get high predictive accuracy.