3.2.3.1 The Architecture of Hybrid Approach

Here we discuss the different approaches of applying Hybridization according to the challenges faced by the researchers where we need to understand the problems and what they are the different possible solutions of it. There are several Hybridization Methods (HM) which are as follows:

1. Weighted: A single recommendation will be provided from the score of several recommendation technique ex-P-Tango systems.
2. Switching: It is a method which switches between different recommendation techniques depending on the situation like Netflix initially content-based is there and after that, it switches to collaborative filtering.

   Paper name	Type of RS	Technique	Evaluation metric	Drawback/Future work
   Panda, S.K., et al. [7]	CF	Normalization based collaborative filtering	1. Recall 2. Precision 3. Mean Absolute Error (MAE) 4. F-Score 5. FMI 6. Root Mean Square Error (RMSE)	1. The rating matrix used in the suggested algorithm is static. 2. The authors have not observed user and item biases in the proposed algorithm. 3. The proposed algorithm is not exhibited using a standard dataset due to the inclusion of rigid users.
   Amara, S., et al. [10]	Content-Based RS	Tagger-User Profile-Tree	1. Precision	1. The tag-based system updates over a while. 2. It shows that profiling through this algorithm is much better than personal and content-based filtering.
   Deng, F., [12]	Utility-Based RS	IU-GA (Item Utility-Genetic Algorithm) extracts the utility functions based on user behavior while browsing not one attribute. This utility function is multiattribute and then the genetic algorithm is used.	1. Precision 2. Recall 3. F1	1. This recommendation system works on limited context online.
   Ullah, F., et al. [21]	Collaborative Filtering	A novel Deep Neural Collaborative Filtering	1. RMSE 2. MAE 3. Similarity measure 4. Predicted Score	1. The side information and other contextual data in the educational service recommendation.
   Feng, C., et al. [22]	Collaborative Filtering	Uses the model that integrates the rating matrix with neighbors information and this model is called the Fusion probability Matrix Factorization model	1. MAE 2. RMSE	1. In the future, the author wants to use the incremental learning approach with the proposed CF model i.e. Fusion based.
   Zarzour, H., et al. [23]	Collaborative Filtering	A trust-based model implemented through collaborative filtering	1. RMSE	1. This collaborative filtering method established on a method called dimensionality reduction which can be paired with a clustering algorithm.
   Mustaqeem, A., Anwar, et al. [24]	Collaborative Filtering	Cluster-Based Modular Recommendation System	1. Precision 2. Recall 3. MAE	1. It dealt with only one type of patients. 2. The dataset will be extended by adding more patients and partitioned based on demographics.
   Zhang, B., et al. [25]	Collaborative Filtering	Residual Convolution CF (Work on Combined approach of PMT + CNN)	1. RMSE	1. Current work is on textual features, in the future latent factor features can also be considered.
   Gu, Y., Zhao, B., et al. [26]	Content-Based RS	An approach that learns (Supervised Learning) simultaneously how to find optimal global term weight and finds the similarity on the multiple text features between user and item profiles.	1. Area under ROC(Receiver operating characteristics) 2. Precision 3. Recall 4. Area under precision-recall curve.	2. The proposed approach improves AUC up to 17% but in the future efficiency can be increased. So this unified framework provides an overall relevance model.
   Zheng, Y., [27]	Utility-Based RS	It uses a Multi-criteria utility-based algorithm.	1. Pearson correlation 2. Cosine similarity 3. Euclidean distance	1. There is an issue called over expectation, which may contribute to finger-grained recommendation models.
   Huang, S.L., et al. [28]	Utility-Based RS	A decomposed and holistic utility-based based method.	1. Accuracy 2. Time Expense	1. In this paper, only 96 students as subjects the limited no of participants decrease the credibility of the research. 2. Utility-based can be compared with collaborative filtering too. 3. Research finds that content-based work on nominal attributes, decomposed one will work better with numerical attributes and holistic will work in different contexts.
   Patro, S.G.K., et al. [29]	Knowledgebased RS	Based on domain knowledge, the preference is created and a learning model is prepared with an adaptive neuro-fuzzy model	1. Mean absolute percentage error 2. Root Mean Square Error 3. Mean Absolute Error 4. Precision 5. F1-score	1. This algorithm works in three steps first one is clustering then decomposition and then prediction any step error may affect the prediction.
   Dong, M., et al. [30]	Knowledge-Based RS	Interactive Knowledge-based Design Recommender System	1. Rate of satisfaction	1. It can be enhanced by learning different design cases.
   Bobadilla, J., et al. [31]	Demographic RS	Obtaining different demographic features from the different latent factors by using the gradient localization method	2. Accuracy	1. This algorithm cannot be able to work on zip code or salary as user demographic feature. 2. Detection of unreported minority users, as well as items, is still one of the limitations which can be solved in the future.
   Yang, C., et al. [32]	Collaborative Filtering	Neural Collaborative filtering approach i.e. Gated and Attentive (GANCF)	1. Hit ratio 2. Normalized Discounted Cumulative Gain (NDCG)	1. In the future, explore the area of social networks and how users trust this platform. 2. The list of interests of the user is realized in the online recommender system.
   Huang, X.Y., et al. [33]	Collaborative Filtering	Logo (Local and global) online CF algorithm	1. RMSE 2. MAE	1. Implicit and explicit feedback can be used in the recommendation. 2. Assumptions for each rating vector can be a mixture of some distributions.
   Cami, B.R., et al. [34]	Content-Based RS	A New Extended Distance Dependent CRP (Chinese Restaurant Process)	1. Precision 2. Recall 3. F1-measure	1. Incorporating a collaborative approach with the current technology provides a richer user profile.
   Bagher, R.C., et al. [35]	Content-Based RS	A Bayesian framework works on the nonparametric area is known as DPMM (Dirichlet Process Mixture model)	1. Precision 2. Recall 3. F1-measure 4. DCG (Discounted cumulative Gain)	1. User rate feedback can be used as a parameter. 2. For improving the diversity measure one can use the CF.

3. Mixed: Different recommendation methods are present and used over the same time.
4. Feature Combination: Using different features that come as the result of different recommendation data and put it into one recommendation algorithm.
5. Cascade: It follows the staged process in which first produces the coarse ranking of user and items as well and refines the recommendation with other techniques.
6. Feature augmentation: It is a technique in which the output of one is the input of another recommendation technique
7. Meta-level: Here the model learned by one recommender is used as input to another.

As we all know that recommender system is increasingly used in the field of e-commerce, movies, music as well as education and health sectors. Hybrid is a solution for all the individual recommender system which comes with certain inbuilt problems [16] As we all know, collaborative filtering has one major problem which is the cold start problem “problem of Recommendation in which no past interaction provided for the user as well as for an item”. The categorization of cold start is of two types which are as follows:

1. User Cold Start
2. Item Cold Start.

Both the above mentioned cold start problems are fixed by using the side information about users and items into a deep neural network. Also, resolves the problem of the linear latent factors, the solution is learning through no-linear latent factors of users and items. This is not the only way to handle the problems of different recommender systems there are [36] which focuses on the item targeting problem means “best as per user interest” so it finds the solution through a hybrid approach in which the Bayesian recommender model is being created which shows the advantages of memory-based and model-based collaborative filtering. Individually, Model-based CF has better fitting operations as well as works when data is missing (data sparsity). Matrix factorization is one of the techniques used for latent factors as well as it solves the problem of cold start. The Hybrid approach comes with several advantages that are useful for solving existing problems:

• a) It increases the interaction between user and item as most as possible for making a better recommendation system.
• b) Researchers work on merging several recommendation models like factor-based, item-based, and user-based collaborative filtering models as a hybrid for artist based recommender system it improves the user-item interaction.
• c) For the item identification problem, the hybrid approach works to propose a model called the “Bayesian inference-based model”.

As the author is focusing on serendipity [37] which is one of the challenges we discussed in an earlier section of the paper, the author provides the hybridization method to face this challenge. This is more subjective than the attribute-based so a large set of data set has been taken and find the following criteria:

1. Content difference: It is defined as a movie that is not searched by the user in past and gets the recommendation.
2. Genre accuracy: For the short term of preference for the movie for the user.
3. Serendipity: It shows both attractiveness and surprise based on Content difference and Genre accuracy.
4. User elasticity: It is the ability to accept the difference from past behavior.
5. Movie elasticity: It is the possibility of being adopted by a different user.
6. Relevance network: It is a network that shows the asymmetric association between different nodes; this node is nothing but the movies and users.

3.2.4.1 Evaluation Measures

How will one find out which recommender system is better than the other? So for that we need appropriate evaluation criteria or metric through which one can understand there are different evaluation metrics like Accuracy, Adaptability, and Reliability which are all not used in all the RS. It depends on the models which evaluation criteria provide the performance analysis. Recommender systems can be evaluated through various evaluation metrics; it is based on the experimental setup for various research studies. It can be understood through these terms:

• Hypothesis: Before starting an experiment, a hypothesis is built up like any algorithm X is better than algorithm Y so we need to be precise about these hypotheses and only use one measure to find and prove this hypothesis is “prediction accuracy”. The researcher doesn’t require any other factor for this hypothesis. The major drawback is the researcher becomes restrictive.
• Controlling Variable: So when according to variable the result may change and we can’t just depend on prediction accuracy because dataset is never static as it changes from time to time and we can’t publish the fact that results differ because of algorithm or dataset so we require more evaluation measures.
• Generalization Power: It is generalizing the experiments based on different datasets. When one will deploy or develop any new algorithm it should not be specific to any one application; it must be beyond that application for that generalization of results are necessary.

How one will decide which recommendation system is to select or better so the answer different parameters that affect the properties of the recommender system is used to evaluate and these measures get affected based on user experience.

Prediction Accuracy: It is one of the properties of RS and it provides more accurate predictions according to the user’s interest. PA evaluates the accuracy of item ratings displayed by users.

Measuring Rating Prediction Accuracy

• a. Root Mean Square Error: When the true ratings are known and the system produces predicted ratings for a test of user-item pairs.
  

  Where, Ŷ1, Ŷ2… Ŷi is predicted values

  Y1, Y2…..Yi is the observed value N is no. of observation.

• b. Mean Absolute Error: The average of all absolute errors in the definition of MAE. An absolute error is a difference between the measured value and the true value.

Where x_i is the measurement, x is the true value.

Where, n = the number of errors, |x_i – x| = the absolute errors

Measuring Usage Predication

A dataset consisting of items that each user used is known as usage prediction and inside these various measures are works based on True Negative, False Positive, False Negative, True positive. Now, these four types are based on the actual value and predicted value as we can see in Table 3.3.

• a. Precision: When fractions of positive number of cases among the total number of positive cases in the system are retrieved then it is known as positive class prediction which is another name of precision. When it predicts yes the person likes cats, how often is it correct?
  

  Conditions	Actual value	Predicted value
  True Negative	False	False
  False Positive	False	True
  False Negative	True	False
  True Positive	True	True

• b. Recall: Number of positive class predictions out of positive class in the dataset. It answers the questions that when it is actually yes the person likes cats, how often does it predict correctly?
  
• c. F-measure: It is a measure that takes both precisions and recalls into account and when one finds out the weighted average of precision and recall then it is known as F-measure and it also defines as a parameter that makes a compromise to be reached concerning precision and recall.

Measuring Ranking of Items

Normalized Distance-Based Performance Measure: When there is a need for information retrieval then this measure is used and it distinguishes correct and incorrect orders of pairs and ties.

3.2.5 Materials and Methods

Recommender System is a part of Data mining. As we see in recommender systems the main focus is on sampling, dimensional reduction, and classification, clustering technique in RS. Also there is association rule discovery and RS is also implemented through deep learning so in this section we understand different algorithm works on the above-mentioned concept.

Sampling is the concept of selecting a small amount of relevant data i.e. subset from a large dataset. This subset is used for training and testing purposes. When a task like model fitting is there then a training dataset is used. When one wants to see how the model generalizes then a testing dataset is used. Various sampling techniques are there like random sampling, stratified sampling. The approach used for sampling is with replacement means items can be removed from the population once they get selected and without replacement means that no item gets removed.

A dimension reduction is a technique that helps to overcome the problem by changing high-dimensional space into lower dimensionality. There is a various method of doing so, PCA, Matrix factorization (like SVD) is one of the most common and powerful techniques of dimensionality reduction.

Denoising is a very important part of pre-processing in which removal of the unwanted effect on data when widening of information is done; in simple words we define unwanted artifacts as noise.

A classifier is of two types: supervised and unsupervised. It is defined as the mapping between label space and feature space. Now here feature means elements characteristics for proper classification and for representing classes label space is used. For example, Hospital RS classifies into two categories i.e. either good or bad based on features that are associated with it.

Instance-Based Classifier: It is defined as the classifier that works on the training and uses it to forecast the labels of unseen cases. It learns the entire set of training and classifies when a new record matches the training example.

kNN classifier is one of the most used classifiers not only in the recommender system but in machine learning too. It is also known as lazy learners and one of the advantages of this technique is that it doesn’t require learning and maintaining a given model and in RS it is used in Collaborative filtering. Nearest Neighbor is used in CF to find like-minded users but this approach is also get challenged by other approaches called Matrix completion.

Decision Tree: It is a type of Nearest Neighbor and as the name suggests it creates a tree based on the target class. A single attribute-value is represented as a Decision node and is tested to find which branch of leaf nodes indicates the target value. Different methods are used for the decision tree like Hunt’s Algorithm. Hunt algorithm works on test conditions implemented on the specified attribute that inclines the observations by their target value. The Decision tree for ranking has also been used.

Rule-Based Classifier works based on various “if...else rules ” and thus this classifier is used to develop the descriptive models and the condition used with “if” is called precedent and the predicted class for each rule works as a consequence. To implement the Rule-based classifier is a classifier that extracts rules directly over different data some examples are Ripper, CN2.

Logistic Regression is the basic probabilistic classification model; it is a type of classifier but then also it is called regression because of the legacy used for linear regression. Linear regression is used for linear equations. In one paper [38] the author uses the regression technique and other nonlinear techniques like AI and made the comparison between which model works better for weather prediction so the researcher can analyze the performance of both linear and non-linear methods. In the regression model, a regular value comes as output but on other hand, the classifier has the output in the form of a class label. Logistic Regression work on the decision boundary many other classification methods that work on a similar concept like SVM (Support Vector Machine).

A Support Vector Machine is a classifier that is a user to find the linear hyperplane which separates the data with maximum margin. Here a linear separation is w.x + b = 0 and the other two are separated by using the class separation function which is based on minimum distance but when the condition arises of non-linearly item then SVM (Support Vector Machine) provides the solution of soft margin classifier by introducing a new variable called “Slack Variable”. Now condition arises of non-linear decision boundary so for that data to change into higher dimensional space it can be done through the help of a mathematical transformation called “Kernel Trick”.

A Bayesian classifier is used to solve the classification problem which works on the concept of a probabilistic framework. Some researchers say that this classifier is the fastest and the gradient boost model is the most accurate algorithm. It is used to represent the uncertainty among the relationship learned from data. It considers each of the attribute and class labels as random variables. It not only uses in collaborative filtering but also in a hybrid approach the example of this is [36] the method calculates or computes the probability through which the user rate the item. When we talk about the hybrid approach it includes all clustering and classification algorithms in a certain way.

Artificial Neural Network is an accumulation [39] of interconnected nodes and these nodes are known as a neuron. The simplest case is the perceptron model when we specify an activation function as a threshold function. Another most common approach is feed-forward ANN.

An Ensemble of Classifiers is a classifier that creates training data from a set of classifiers and does the prediction based on class labels. The two most common techniques used in ensemble learning is Bagging and Boosting.

Bagging is defined as performing the sampling with reinstatement and built the classifier on each step of bootstrap.

Boosting is an adaptive change of distribution in training data by giving importance to the previous misclassified records. In this initially, all records are of equal weights, and then wrongly classified records have higher weights have their weights increased, and those which are correctly classified with lessen weights. Adaboost is one of the algorithms that is used for boosting an ensemble classifier. This classifier is most commonly used in the hybrid approach RS.

In this clustering technique, the researcher [2] proposed a new algorithm that works on the semantics information of the web page, and pages are set into clusters by using a clustering algorithm and builds upon the similarity of semantics with the user. In this, the author provides the user agent which works as an interface where the user can input and gets output. The knowledge base is also created to classify the information as the user model there is a behavior model that precisely suggests the behavior of each user and all this model help in finding a better recommendation system.

In this paper, [24] the author gives the new method applied on supervised data in four different classes of disease retrieved by using the Kmean clustering and this method is implemented on each partition separately, and once the query patient is partitioned into the correct class of disease then it requires the similarity scores for reduced sub-clusters so it means that a separate recommendation is used for each class of disease and the whole process is known as modularization.

In this paper [40] the author suggests the solution for the issue of classifying the text to a discrete set of predefined categories it helps in identifying the categorical terms present in emails for this author uses the LDA modeling approach. The author [41] built the two datasets for evaluation: first one is transaction data and the other one is demographic data (personal data of user transaction data is calculated through similarity matrix and demographic data works on the cluster-product rating matrix by using K mean clustering technique. Fuzzy logic is an application-based approach that is used for classification [42]; the author proposed the idea for text summarization, fuzzy logic is used to normalize the unspecific reasoning of human ability. As the author proposed the Fuzzy logic based on the triangular member function which maps the input space points to membership functions and by using this method improvement takes place on the present model which is evaluated through different measures like precision and recall.

3.2.6 Comparative Analysis With Traditional Recommender System

When we say comparative analysis it means that what are the problems in one system and how it is solved by any other system through which better RS we get and we understand it through various evaluations metric in Table 3.4. Dataset of different types shows different results so we can’t say that if one algorithm is working in one data set so it will work for another so the size of the data set also matters at the time of implementing any new algorithm.

3.2.7 Practical Implications

As per Research Questions (RQ), we find the answers to these questions:

1. RQ1 (Answer): When we talk about collaborative filtering then there are various challenges we face in an existing system like the cold start problem as well as gray sheep problem (discussed in challenges) are present prominently and if we say what are the practical implications are there so the solution is tagging as well as using content-based or feature-related extraction and make a recommendation through that main problem with collaborative is it depends on rating matrix either it is user-based or item-based but recommendations explored by many researchers says that accuracy and precision not only based on this, it depends on the ranking scheme and user profile through this we get better results so by combining all this approach in a different way hybrid approach arises and all the paper we read hybrid gives a better result than the individual traditional approach.
2. RQ2 (Answer): If we talk about existing solutions through the existing traditional approach of RS then for cold start content-based work as a solution, for data-sparsity one can provide the solution like building a trust-based model, one of the challenges of diversity is solved through the hybrid system [36]. The scalability problem is solved through a context-aware strategy.

   

3. RQ3 (Answer): After exploring and understanding the above two research we concluded that the hybrid approach is one of the most flexible and by taking a different approach and combined in a distinguished way and gives a better result.

3.2.8 Conclusion & Future Work

The hybrid approach solves the problem of the traditional filtering technique and improves the effectiveness of recommendation by using different classifiers and clustering techniques. Many significant works have been done in this field to give users a recommendation based on their interests. In various traditional approaches there are some drawbacks like collaborative filtering suffers from data sparsity, cold start and scalability problems in demographic, there is a problem of breaching user privacy and in utility-based how to find that which product is more utilized for user and knowledge base create a complex system to understand and maintain. In all these, we have various other issues regarding dataset not in all dataset proposed algorithm works appropriately and accurately. Training and testing are one of the fields that may change the experiment results so all these will become the discussion point shortly that how researcher decides what the percentage of testing data is and what the percentage of training data will be present. Also Hybrid approach includes the deep neural network concept which also provides better results in terms of precision and recall.

A hybrid approach is the most flexible and most used approach of research which shows how many different ways one can use the traditional approaches and solve all the disadvantages of an individual approach but the problem is that how one will find that which two combination works better than the other.