Recommender system

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search
Recommender systems
Concepts
Methods and challenges
Implementations
Research

A recommender system or a recommendation system (sometimes replacing "system" with a synonym such as platform or engine) is a subclass of information filtering system that seeks to predict the "rating" or "preference" a user would give to an item.[1][2]

Recommender systems are utilized in a variety of areas including movies, music, news, books, research articles, search queries, social tags, and products in general. There are also recommender systems for experts,[3] collaborators,[4] jokes, restaurants, garments, financial services,[5] life insurance, romantic partners (online dating), and Twitter pages.[6]

Overview[edit]

Recommender systems typically produce a list of recommendations in one of two ways – through collaborative filtering or through content-based filtering (also known as the personality-based approach).[7] Collaborative filtering approaches build a model from a user's past behaviour (items previously purchased or selected and/or numerical ratings given to those items) as well as similar decisions made by other users. This model is then used to predict items (or ratings for items) that the user may have an interest in.[8] Content-based filtering approaches utilize a series of discrete characteristics of an item in order to recommend additional items with similar properties.[9] These approaches are often combined (see Hybrid Recommender Systems).

The differences between collaborative and content-based filtering can be demonstrated by comparing two popular music recommender systems – Last.fm and Pandora Radio.

  • Last.fm creates a "station" of recommended songs by observing what bands and individual tracks the user has listened to on a regular basis and comparing those against the listening behavior of other users. Last.fm will play tracks that do not appear in the user's library, but are often played by other users with similar interests. As this approach leverages the behavior of users, it is an example of a collaborative filtering technique.
  • Pandora uses the properties of a song or artist (a subset of the 400 attributes provided by the Music Genome Project) to seed a "station" that plays music with similar properties. User feedback is used to refine the station's results, deemphasizing certain attributes when a user "dislikes" a particular song and emphasizing other attributes when a user "likes" a song. This is an example of a content-based approach.

Each type of system has its strengths and weaknesses. In the above example, Last.fm requires a large amount of information about a user to make accurate recommendations. This is an example of the cold start problem, and is common in collaborative filtering systems.[10][11][12] Whereas Pandora needs very little information to start, it is far more limited in scope (for example, it can only make recommendations that are similar to the original seed).

Recommender systems are a useful alternative to search algorithms since they help users discover items they might not have found otherwise. Of note, recommender systems are often implemented using search engines indexing non-traditional data.

Recommender systems were first mentioned in a technical report as a "digital bookshelf" in 1990 by Jussi Karlgren at Columbia University,[13] and implemented at scale and worked through in technical reports and publications from 1994 onwards by Jussi Karlgren, then at SICS,[14] [15] and research groups led by Pattie Maes at MIT,[16] Will Hill at Bellcore,[17] and Paul Resnick, also at MIT[18] [19] whose work with GroupLens was awarded the 2010 ACM Software Systems Award.

Montaner provided the first overview of recommender systems from an intelligent agent perspective.[20] Adomavicius provided a new, alternate overview of recommender systems.[21] Herlocker provides an additional overview of evaluation techniques for recommender systems,[22] and Beel et al. discussed the problems of offline evaluations.[23] Beel et al. have also provided literature surveys on available research paper recommender systems and existing challenges.[24][25][26]

Recommender systems have been the focus of several granted patents. [27][28][29][30][31]

Approaches[edit]

Collaborative filtering[edit]

Main article: Collaborative filtering

One approach to the design of recommender systems that has wide use is collaborative filtering.[32] Collaborative filtering methods are based on collecting and analyzing a large amount of information on users’ behaviors, activities or preferences and predicting what users will like based on their similarity to other users. A key advantage of the collaborative filtering approach is that it does not rely on machine analyzable content and therefore it is capable of accurately recommending complex items such as movies without requiring an "understanding" of the item itself. Many algorithms have been used in measuring user similarity or item similarity in recommender systems. For example, the k-nearest neighbor (k-NN) approach[33] and the Pearson Correlation as first implemented by Allen.[34]

Collaborative filtering is based on the assumption that people who agreed in the past will agree in the future, and that they will like similar kinds of items as they liked in the past.

When building a model from a user's behavior, a distinction is often made between explicit and implicit forms of data collection.

Examples of explicit data collection include the following:

  • Asking a user to rate an item on a sliding scale.
  • Asking a user to search.
  • Asking a user to rank a collection of items from favorite to least favorite.
  • Presenting two items to a user and asking him/her to choose the better one of them.
  • Asking a user to create a list of items that he/she likes.

Examples of implicit data collection include the following:

  • Observing the items that a user views in an online store.
  • Analyzing item/user viewing times.[35]
  • Keeping a record of the items that a user purchases online.
  • Obtaining a list of items that a user has listened to or watched on his/her computer.
  • Analyzing the user's social network and discovering similar likes and dislikes.

The recommender system compares the collected data to similar and dissimilar data collected from others and calculates a list of recommended items for the user. Several commercial and non-commercial examples are listed in the article on collaborative filtering systems.

One of the most famous examples of collaborative filtering is item-to-item collaborative filtering (people who buy x also buy y), an algorithm popularized by Amazon.com's recommender system.[36] Other examples include:

  • As previously detailed, Last.fm recommends music based on a comparison of the listening habits of similar users, while Readgeek compares books ratings for recommendations.
  • Facebook, MySpace, LinkedIn, and other social networks use collaborative filtering to recommend new friends, groups, and other social connections (by examining the network of connections between a user and their friends).[1] Twitter uses many signals and in-memory computations for recommending to its users whom they should "follow."[6]

Collaborative filtering approaches often suffer from three problems: cold start, scalability, and sparsity.[37]

  • Cold start: These systems often require a large amount of existing data on a user in order to make accurate recommendations.[10][11]
  • Scalability: In many of the environments in which these systems make recommendations, there are millions of users and products. Thus, a large amount of computation power is often necessary to calculate recommendations.
  • Sparsity: The number of items sold on major e-commerce sites is extremely large. The most active users will only have rated a small subset of the overall database. Thus, even the most popular items have very few ratings.

A particular type of collaborative filtering algorithm uses matrix factorization, a low-rank matrix approximation technique.[38][39][40]

Collaborative filtering methods are classified as memory-based and model based collaborative filtering. A well-known example of memory-based approaches is user-based algorithm[41] and that of model-based approaches is Kernel-Mapping Recommender.[42]

Content-based filtering[edit]

Another common approach when designing recommender systems is content-based filtering. Content-based filtering methods are based on a description of the item and a profile of the user’s preferences.[43][44]

In a content-based recommender system, keywords are used to describe the items and a user profile is built to indicate the type of item this user likes. In other words, these algorithms try to recommend items that are similar to those that a user liked in the past (or is examining in the present). In particular, various candidate items are compared with items previously rated by the user and the best-matching items are recommended. This approach has its roots in information retrieval and information filtering research.

To abstract the features of the items in the system, an item presentation algorithm is applied. A widely used algorithm is the tf–idf representation (also called vector space representation).

To create a user profile, the system mostly focuses on two types of information:

1. A model of the user's preference.

2. A history of the user's interaction with the recommender system.

Basically, these methods use an item profile (i.e., a set of discrete attributes and features) characterizing the item within the system. The system creates a content-based profile of users based on a weighted vector of item features. The weights denote the importance of each feature to the user and can be computed from individually rated content vectors using a variety of techniques. Simple approaches use the average values of the rated item vector while other sophisticated methods use machine learning techniques such as Bayesian Classifiers, cluster analysis, decision trees, and artificial neural networks in order to estimate the probability that the user is going to like the item.[45]

Direct feedback from a user, usually in the form of a like or dislike button, can be used to assign higher or lower weights on the importance of certain attributes (using Rocchio classification or other similar techniques).

A key issue with content-based filtering is whether the system is able to learn user preferences from users' actions regarding one content source and use them across other content types. When the system is limited to recommending content of the same type as the user is already using, the value from the recommendation system is significantly less than when other content types from other services can be recommended. For example, recommending news articles based on browsing of news is useful, but would be much more useful when music, videos, products, discussions etc. from different services can be recommended based on news browsing.

Pandora Radio is an example of a content-based recommender system that plays music with similar characteristics to that of a song provided by the user as an initial seed. There are also a large number of content-based recommender systems aimed at providing movie recommendations, a few such examples include Rotten Tomatoes, Internet Movie Database, Jinni, Rovi Corporation, and Jaman. Document related recommender systems aim at providing document recommendations to knowledge workers. Public health professionals have been studying recommender systems to personalize health education and preventative strategies.[46][47]

Hybrid recommender systems[edit]

Recent research has demonstrated that a hybrid approach, combining collaborative filtering and content-based filtering could be more effective in some cases. Hybrid approaches can be implemented in several ways: by making content-based and collaborative-based predictions separately and then combining them; by adding content-based capabilities to a collaborative-based approach (and vice versa); or by unifying the approaches into one model (see[21] for a complete review of recommender systems). Several studies empirically compare the performance of the hybrid with the pure collaborative and content-based methods and demonstrate that the hybrid methods can provide more accurate recommendations than pure approaches. These methods can also be used to overcome some of the common problems in recommender systems such as cold start and the sparsity problem.

Netflix is a good example of the use of hybrid recommender systems.[48] The website makes recommendations by comparing the watching and searching habits of similar users (i.e., collaborative filtering) as well as by offering movies that share characteristics with films that a user has rated highly (content-based filtering).

A variety of techniques have been proposed as the basis for recommender systems: collaborative, content-based, knowledge-based, and demographic techniques. Each of these techniques has known shortcomings, such as the well known cold-start problem for collaborative and content-based systems (what to do with new users with few ratings) and the knowledge engineering bottleneck[49] in knowledge-based approaches. A hybrid recommender system is one that combines multiple techniques together to achieve some synergy between them.

  • Collaborative: The system generates recommendations using only information about rating profiles for different users or items. Collaborative systems locate peer users/items with a rating history similar to the current user or item and generate recommendations using this neighborhood. The user-based and the item based nearest neighbor algorithms can be combined to deal with the cold start problem and improve recommendation results.[50]
  • Content-based: The system generates recommendations from two sources: the features associated with products and the ratings that a user has given them. Content-based recommenders treat recommendation as a user-specific classification problem and learn a classifier for the user's likes and dislikes based on product features.
  • Demographic: A demographic recommender provides recommendations based on a demographic profile of the user. Recommended products can be produced for different demographic niches, by combining the ratings of users in those niches.
  • Knowledge-based: A knowledge-based recommender suggests products based on inferences about a user’s needs and preferences. This knowledge will sometimes contain explicit functional knowledge about how certain product features meet user needs.[51][52]

The term hybrid recommender system is used here to describe any recommender system that combines multiple recommendation techniques together to produce its output. There is no reason why several different techniques of the same type could not be hybridized, for example, two different content-based recommenders could work together, and a number of projects have investigated this type of hybrid: NewsDude, which uses both naive Bayes and k-nearest neighbors classifiers in its news recommendations is just one example.[51]

Seven hybridization techniques:

  • Weighted: The score of different recommendation components are combined numerically.
  • Switching: The system chooses among recommendation components and applies the selected one.
  • Mixed: Recommendations from different recommenders are presented together to give the recommendation.
  • Feature Combination: Features derived from different knowledge sources are combined together and given to a single recommendation algorithm.
  • Feature Augmentation: One recommendation technique is used to compute a feature or set of features, which is then part of the input to the next technique.
  • Cascade: Recommenders are given strict priority, with the lower priority ones breaking ties in the scoring of the higher ones.
  • Meta-level: One recommendation technique is applied and produces some sort of model, which is then the input used by the next technique.[51]

Beyond accuracy[edit]

Typically, research on recommender systems is concerned about finding the most accurate recommendation algorithms. However, there are a number of factors that are also important.

  • Diversity – Users tend to be more satisfied with recommendations when there is a higher intra-list diversity, e.g. items from different artists.[53]
  • Recommender persistence – In some situations, it is more effective to re-show recommendations,[54] or let users re-rate items,[55] than showing new items. There are several reasons for this. Users may ignore items when they are shown for the first time, for instance, because they had no time to inspect the recommendations carefully.
  • Privacy – Recommender systems usually have to deal with privacy concerns[56] because users have to reveal sensitive information. Building user profiles using collaborative filtering can be problematic from a privacy point of view. Many European countries have a strong culture of data privacy, and every attempt to introduce any level of user profiling can result in a negative customer response. A number of privacy issues arose around the dataset offered by Netflix for the Netflix Prize competition. Although the data sets were anonymized in order to preserve customer privacy, in 2007 two researchers from the University of Texas were able to identify individual users by matching the data sets with film ratings on the Internet Movie Database.[57] As a result, in December 2009, an anonymous Netflix user sued Netflix in Doe v. Netflix, alleging that Netflix had violated United States fair trade laws and the Video Privacy Protection Act by releasing the datasets.[58] This led in part to the cancellation of a second Netflix Prize competition in 2010.[59] Much research has been conducted on ongoing privacy issues in this space. Ramakrishnan et al. have conducted an extensive overview of the trade-offs between personalization and privacy and found that the combination of weak ties (an unexpected connection that provides serendipitous recommendations) and other data sources can be used to uncover identities of users in an anonymized dataset.[60]
  • User demographics – Beel et al. found that user demographics may influence how satisfied users are with recommendations.[61] In their paper they show that elderly users tend to be more interested in recommendations than younger users.
  • Robustness – When users can participate in the recommender system, the issue of fraud must be addressed.[62]
  • Serendipity – Serendipity is a measure of "how surprising the recommendations are".[63] For instance, a recommender system that recommends milk to a customer in a grocery store might be perfectly accurate, but it is not a good recommendation because it is an obvious item for the customer to buy.
  • Trust – A recommender system is of little value for a user if the user does not trust the system.[64] Trust can be built by a recommender system by explaining how it generates recommendations, and why it recommends an item.
  • Labelling – User satisfaction with recommendations may be influenced by the labeling of the recommendations.[65] For instance, in the cited study click-through rate (CTR) for recommendations labeled as "Sponsored" were lower (CTR=5.93%) than CTR for identical recommendations labeled as "Organic" (CTR=8.86%). Recommendations with no label performed best (CTR=9.87%) in that study.

Mobile recommender systems[edit]

Further information: Location based recommendation

One growing area of research in the area of recommender systems is mobile recommender systems. With the increasing ubiquity of internet-accessing smart phones, it is now possible to offer personalized, context-sensitive recommendations. This is a particularly difficult area of research as mobile data is more complex than data that recommender systems often have to deal with (it is heterogeneous, noisy, requires spatial and temporal auto-correlation, and has validation and generality problems[66]). Additionally, mobile recommender systems suffer from a transplantation problem – recommendations may not apply in all regions (for instance, it would be unwise to recommend a recipe in an area where all of the ingredients may not be available).

One example of a mobile recommender system is one that offers potentially profitable driving routes for taxi drivers in a city.[66] This system takes input data in the form of GPS traces of the routes that taxi drivers took while working, which include location (latitude and longitude), time stamps, and operational status (with or without passengers). It uses this data to recommend a list of pickup points along a route, with the goal of optimizing occupancy times and profits. This type of system is obviously location-dependent, and since it must operate on a handheld or embedded device, the computation and energy requirements must remain low.

Another example of mobile recommendation is what (Bouneffouf et al., 2012) developed for professional users. Using GPS traces of the user and his agenda, it suggests suitable information depending on his situation and interests. The system uses machine learning techniques and reasoning processes in order to dynamically adapt the mobile recommender system to the evolution of the user’s interest. The author called his algorithm hybrid-ε-greedy.[67]

Mobile recommendation systems have also been successfully built using the "Web of Data" as a source for structured information. A good example of such system is SMARTMUSEUM[68] The system uses semantic modelling, information retrieval, and machine learning techniques in order to recommend content matching user interests, even when presented with sparse or minimal user data.

There are three factors that could affect the mobile recommender systems and the accuracy of prediction results: the context, the recommendation method and privacy.[69]

Risk-aware recommender systems[edit]

The majority of existing approaches to recommender systems focus on recommending the most relevant content to users using contextual information and do not take into account the risk of disturbing the user in specific situation. However, in many applications, such as recommending personalized content, it is also important to consider the risk of upsetting the user so as not to push recommendations in certain circumstances, for instance, during a professional meeting, early morning, or late at night. Therefore, the performance of the recommender system depends in part on the degree to which it has incorporated the risk into the recommendation process.

Risk definition[edit]

"The risk in recommender systems is the possibility to disturb or to upset the user which leads to a bad answer of the user".[70]

In response to these challenges, the authors in DRARS, A Dynamic Risk-Aware Recommender System[70] have developed a dynamic risk sensitive recommendation system called DRARS (Dynamic Risk-Aware Recommender System), which models the context-aware recommendation as a bandit problem. This system combines a content-based technique and a contextual bandit algorithm. They have shown that DRARS improves the Upper Confidence Bound (UCB) policy, the currently available best algorithm, by calculating the most optimal exploration value to maintain a trade-off between exploration and exploitation based on the risk level of the current user's situation. The authors conducted experiments in an industrial context with real data and real users and have shown that taking into account the risk level of users' situations significantly increased the performance of the recommender systems.

The Netflix Prize[edit]

Main article: Netflix Prize

One of the events that energized research in recommender systems was the Netflix Prize. From 2006 to 2009, Netflix sponsored a competition, offering a grand prize of $1,000,000 to the team that could take an offered dataset of over 100 million movie ratings and return recommendations that were 10% more accurate than those offered by the company's existing recommender system. This competition energized the search for new and more accurate algorithms. On 21 September 2009, the grand prize of US$1,000,000 was given to the BellKor's Pragmatic Chaos team using tiebreaking rules.[71]

The most accurate algorithm in 2007 used an ensemble method of 107 different algorithmic approaches, blended into a single prediction:[72]

Predictive accuracy is substantially improved when blending multiple predictors. Our experience is that most efforts should be concentrated in deriving substantially different approaches, rather than refining a single technique. Consequently, our solution is an ensemble of many methods.

Many benefits accrued to the web due to the Netflix project. Some teams have taken their technology and applied it to other markets. Some members from the team that finished second place founded Gravity R&D, a recommendation engine that's active in the RecSys community.[71][73] 4-Tell, Inc. created a Netflix project–derived solution for ecommerce websites.

A second contest was planned, but was ultimately canceled in response to an ongoing lawsuit and concerns from the Federal Trade Commission.[59]

Performance measures[edit]

Evaluation is important in assessing the effectiveness of recommendation algorithms. The commonly used metrics are the mean squared error and root mean squared error, the latter having been used in the Netflix Prize. The information retrieval metrics such as precision and recall or DCG are useful to assess the quality of a recommendation method. Recently, diversity, novelty, and coverage are also considered as important aspects in evaluation.[74] However, many of the classic evaluation measures are highly criticized.[75] Often, results of so-called offline evaluations do not correlate with actually assessed user-satisfaction.[76] The authors conclude "we would suggest treating results of offline evaluations [i.e. classic performance measures] with skepticism".

Multi-criteria recommender systems[edit]

Multi-criteria recommender systems (MCRS) can be defined as recommender systems that incorporate preference information upon multiple criteria. Instead of developing recommendation techniques based on a single criterion values, the overall preference of user u for the item i, these systems try to predict a rating for unexplored items of u by exploiting preference information on multiple criteria that affect this overall preference value. Several researchers approach MCRS as a multi-criteria decision making (MCDM) problem, and apply MCDM methods and techniques to implement MCRS systems.[77] See this chapter[78] for an extended introduction.

Opinion-based recommender systems[edit]

In some cases, users are allowed to leave text review or feedback on the items. These user-generated text are implicit data for the recommender system because they are potentially rich resource of both feature/aspects of the item, and the user’s evaluation/sentiment to the item. Features extracted from the user-generated reviews are improved meta-data of items, because as they also reflect aspects of the item like meta-data, extracted features are widely concerned by the users. Sentiments extracted from the reviews can be seen as user’s rating scores on the corresponding features. Popular approaches of opinion-based recommender system utilize various techniques including text mining, information retrieval and sentiment analysis (see also Multimodal sentiment analysis).

Recommender system evaluation[edit]

To measure the effectiveness of recommender systems, and compare different approaches, three types of evaluations are available: user studies, online evaluations (A/B tests), and offline evaluations.[23] User studies are rather small scale. A few dozens or hundreds of users are presented recommendations created by different recommendation approaches, and then the users judge, which recommendations are best. In A/B tests, recommendations are shown to typically thousands of users of a real product, and the recommender system randomly picks at least two different recommendation approaches to generate recommendations. The effectiveness is measured with implicit measures of effectiveness such as conversion rate or click-through rate. Offline evaluations are based on historic data, e.g. a dataset that contains information about how users previously rated movies.[79] The effectiveness of recommendation approaches is then measured based on how well a recommendation approach can predict the users' ratings in the dataset. While a rating is an explicit expression of whether a user liked a movie, such information is not available in all domains. For instance, in the domain of citation recommender systems, users typically do not rate a citation or recommended article. In such cases, offline evaluations may use implicit measures of effectiveness. For instance, it may be assumed that a recommender system is effective that is able to recommend as many articles as possible that are contained in a research article's reference list. However, this kind of offline evaluations is seen critical by many researchers.[80][81][23] For instance, it has been shown that results of offline evaluations have low correlation with results from user studies or A/B tests.[81][82] A dataset popular for offline evaluation has been shown to contain duplicate data and thus to lead to wrong conclusions in the evaluation of algorithms.[83]

Reproducibility in recommender system research[edit]

In recent years there was a growing understanding in the community that lots of previous research had little impact on the practical application of recommender systems. Ekstrand, Konstan, et al. criticize that "it is currently difficult to reproduce and extend recommender systems research results,” and that evaluations are “not handled consistently".[84] Konstan and Adomavicius conclude that "the Recommender Systems research community is facing a crisis where a significant number of papers present results that contribute little to collective knowledge […] often because the research lacks the […] evaluation to be properly judged and, hence, to provide meaningful contributions.".[85] As a consequence, lots of research about recommender systems can be considered as not reproducible.[86] Hence, operators of recommender systems find little guidance in the current research for answering the question, which recommendation approaches to use in a recommender systems. Said & Bellogín conducted a study of recent papers published in the field, as well as benchmarked some of the most popular frameworks for recommendation and found large inconsistencies in results, even when the same algorithms and data sets were used.[87] Some researchers demonstrated that minor variations in the recommendation algorithms or scenarios led to strong changes in the effectiveness of a recommender system. They conclude that seven actions are necessary to improve the current situation:[86] "(1) survey other research fields and learn from them, (2) find a common understanding of reproducibility, (3) identify and understand the determinants that affect reproducibility, (4) conduct more comprehensive experiments (5) modernize publication practices, (6) foster the development and use of recommendation frameworks, and (7) establish best-practice guidelines for recommender-systems research."

While reproducibility has not been considered for a long time in the recommender-system community, this aspects is much more considered recently, with several workshops and conferences focusing on reproducibility in recommender system research.[88]

Anticipatory design[edit]

Anticipatory design is the combination of Internet of Things, User experience design and machine learning. Anticipatory design differs from conventional design, in that within anticipatory design the goal is to simplify the process and minimize difficulty by making decisions on behalf of the users. Examples of services that use various levels of anticipatory designs are: Amazon and Netflix production recommendation which recommend products based on previous behavior, the mobile application Peapod which uses a recommendation engine to allow users to fill their shopping basket based on previous orders, and the Nest thermostat which predicts the perfect room temperature based on user input and time of day.[89]

See also[edit]

References[edit]

  1. ^ a b Francesco Ricci and Lior Rokach and Bracha Shapira, Introduction to Recommender Systems Handbook, Recommender Systems Handbook, Springer, 2011, pp. 1-35
  2. ^ "Facebook, Pandora Lead Rise of Recommendation Engines - TIME". TIME.com. 27 May 2010. Retrieved 1 June 2015.
  3. ^ H. Chen, A. G. Ororbia II, C. L. Giles ExpertSeer: a Keyphrase Based Expert Recommender for Digital Libraries, in arXiv preprint 2015
  4. ^ H. Chen, L. Gou, X. Zhang, C. Giles Collabseer: a search engine for collaboration discovery, in ACM/IEEE Joint Conference on Digital Libraries (JCDL) 2011
  5. ^ Alexander Felfernig, Klaus Isak, Kalman Szabo, Peter Zachar, The VITA Financial Services Sales Support Environment, in AAAI/IAAI 2007, pp. 1692-1699, Vancouver, Canada, 2007.
  6. ^ a b Pankaj Gupta, Ashish Goel, Jimmy Lin, Aneesh Sharma, Dong Wang, and Reza Bosagh Zadeh WTF:The who-to-follow system at Twitter, Proceedings of the 22nd international conference on World Wide Web
  7. ^ Hosein Jafarkarimi; A.T.H. Sim and R. Saadatdoost A Naïve Recommendation Model for Large Databases, International Journal of Information and Education Technology, June 2012
  8. ^ Prem Melville and Vikas Sindhwani, Recommender Systems, Encyclopedia of Machine Learning, 2010.
  9. ^ R. J. Mooney & L. Roy (1999). Content-based book recommendation using learning for text categorization. In Workshop Recom. Sys.: Algo. and Evaluation.
  10. ^ a b Rubens, Neil; Elahi, Mehdi; Sugiyama, Masashi; Kaplan, Dain (2016). "Active Learning in Recommender Systems". In Ricci, Francesco; Rokach, Lior; Shapira, Bracha. Recommender Systems Handbook (2 ed.). Springer US. doi:10.1007/978-1-4899-7637-6_24. ISBN 978-1-4899-7637-6.
  11. ^ a b Elahi, Mehdi; Ricci, Francesco; Rubens, Neil (2016). "A survey of active learning in collaborative filtering recommender systems". Computer Science Review. 20: 29–50. doi:10.1016/j.cosrev.2016.05.002.
  12. ^ Andrew I. Schein, Alexandrin Popescul, Lyle H. Ungar, David M. Pennock (2002). Methods and Metrics for Cold-Start Recommendations. Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002). New York City, New York: ACM. pp. 253–260. ISBN 1-58113-561-0. Retrieved 2008-02-02.CS1 maint: Multiple names: authors list (link)
  13. ^ Karlgren, Jussi. 1990. "An Algebra for Recommendations." Syslab Working Paper 179 (1990).
  14. ^ Karlgren, Jussi. "Newsgroup Clustering Based On User Behavior-A Recommendation Algebra." SICS Research Report (1994).
  15. ^ Karlgren, Jussi (October 2017). "A digital bookshelf: original work on recommender systems". Retrieved 27 October 2017.
  16. ^ Shardanand, Upendra, and Pattie Maes. "Social information filtering: algorithms for automating “word of mouth”." In Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 210-217. ACM Press/Addison-Wesley Publishing Co., 1995.
  17. ^ Hill, Will, Larry Stead, Mark Rosenstein, and George Furnas. "Recommending and evaluating choices in a virtual community of use." In Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 194-201. ACM Press/Addison-Wesley Publishing Co., 1995.
  18. ^ Resnick, Paul, Neophytos Iacovou, Mitesh Suchak, Peter Bergström, and John Riedl. "GroupLens: an open architecture for collaborative filtering of netnews." In Proceedings of the 1994 ACM conference on Computer supported cooperative work, pp. 175-186. ACM, 1994.
  19. ^ Resnick, Paul, and Hal R. Varian. "Recommender systems." Communications of the ACM 40, no. 3 (1997): 56-58.
  20. ^ Montaner, M.; Lopez, B.; de la Rosa, J. L. (June 2003). "A Taxonomy of Recommender Agents on the Internet". Artificial Intelligence Review. 19 (4): 285–330. doi:10.1023/A:1022850703159.
  21. ^ a b Adomavicius, G.; Tuzhilin, A. (June 2005). "Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions". IEEE Transactions on Knowledge and Data Engineering. 17 (6): 734–749. CiteSeerX 10.1.1.107.2790. doi:10.1109/TKDE.2005.99.
  22. ^ Herlocker, J. L.; Konstan, J. A.; Terveen, L. G.; Riedl, J. T. (January 2004). "Evaluating collaborative filtering recommender systems". ACM Trans. Inf. Syst. 22 (1): 5–53. CiteSeerX 10.1.1.78.8384. doi:10.1145/963770.963772.
  23. ^ a b c Beel, J.; Genzmehr, M.; Gipp, B. (October 2013). "A Comparative Analysis of Offline and Online Evaluations and Discussion of Research Paper Recommender System Evaluation" (PDF). Proceedings of the Workshop on Reproducibility and Replication in Recommender Systems Evaluation (RepSys) at the ACM Recommender System Conference (RecSys).
  24. ^ Beel, J.; Langer, S.; Genzmehr, M.; Gipp, B.; Breitinger, C. (October 2013). "Research Paper Recommender System Evaluation: A Quantitative Literature Survey" (PDF). Proceedings of the Workshop on Reproducibility and Replication in Recommender Systems Evaluation (RepSys) at the ACM Recommender System Conference (RecSys).
  25. ^ Beel, J.; Gipp, B.; Langer, S.; Breitinger, C. (26 July 2015). "Research Paper Recommender Systems: A Literature Survey". International Journal on Digital Libraries. 17 (4): 305–338. doi:10.1007/s00799-015-0156-0.
  26. ^ Waila, P.; Singh, V.; Singh, M. (26 April 2016). "A Scientometric Analysis of Research in Recommender Systems" (PDF). Journal of Scientometric Research. 5: 71–84. doi:10.5530/jscires.5.1.10.
  27. ^ Stack, Charles. "System and method for providing recommendation of goods and services based on recorded purchasing history." U.S. Patent 7,222,085, issued May 22, 2007.
  28. ^ Herz, Frederick SM. "Customized electronic newspapers and advertisements." U.S. Patent 7,483,871, issued January 27, 2009.
  29. ^ Herz, Frederick, Lyle Ungar, Jian Zhang, and David Wachob. "System and method for providing access to data using customer profiles." U.S. Patent 8,056,100, issued November 8, 2011.
  30. ^ Harbick, Andrew V., Ryan J. Snodgrass, and Joel R. Spiegel. "Playlist-based detection of similar digital works and work creators." U.S. Patent 8,468,046, issued June 18, 2013.
  31. ^ Linden, Gregory D., Brent Russell Smith, and Nida K. Zada. "Automated detection and exposure of behavior-based relationships between browsable items." U.S. Patent 9,070,156, issued June 30, 2015.
  32. ^ John S. Breese; David Heckerman & Carl Kadie (1998). Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence (UAI'98).
  33. ^ Sarwar, B.; Karypis, G.; Konstan, J.; Riedl, J. (2000). "Application of Dimensionality Reduction in Recommender System A Case Study",
  34. ^ Allen, R.B. (1990). "User Models: Theory, Method, Practice". International J. Man-Machine Studies.
  35. ^ Parsons, J.; Ralph, P.; Gallagher, K. (July 2004). "Using viewing time to infer user preference in recommender systems". AAAI Workshop in Semantic Web Personalization, San Jose, California.
  36. ^ Collaborative Recommendations Using Item-to-Item Similarity Mappings Archived 2015-03-16 at the Wayback Machine
  37. ^ Sanghack Lee and Jihoon Yang and Sung-Yong Park, Discovery of Hidden Similarity on Collaborative Filtering to Overcome Sparsity Problem, Discovery Science, 2007.
  38. ^ I. Markovsky, Low-Rank Approximation: Algorithms, Implementation, Applications, Springer, 2012, ISBN 978-1-4471-2226-5
  39. ^ Takács, G.; Pilászy, I.; Németh, B.; Tikk, D. (March 2009). "Scalable Collaborative Filtering Approaches for Large Recommender Systems" (PDF). Journal of Machine Learning Research. 10: 623–656
  40. ^ Rennie, J.; Srebro, N. (2005). Luc De Raedt, Stefan Wrobel, ed. Fast Maximum Margin Matrix Factorization for Collaborative Prediction (PDF). Proceedings of the 22nd Annual International Conference on Machine Learning. ACM Press.
  41. ^ Breese, John S.; Heckerman, David; Kadie, Carl (1998). Empirical Analysis of Predictive Algorithms for Collaborative Filtering (PDF) (Report). Microsoft Research.
  42. ^ Ghazanfar, Mustansar Ali; Prügel-Bennett, Adam; Szedmak, Sandor (2012-11-15). "Kernel-Mapping Recommender system algorithms". Information Sciences. 208: 81–104. CiteSeerX 10.1.1.701.7729. doi:10.1016/j.ins.2012.04.012. Retrieved 1 June 2015.
  43. ^ Aggarwal, Charu C. (2016). Recommender Systems: The Textbook. Springer. ISBN 9783319296579.
  44. ^ Peter Brusilovsky (2007). The Adaptive Web. p. 325. ISBN 978-3-540-72078-2.
  45. ^ Blanda, Stephanie (May 25, 2015). "Online Recommender Systems – How Does a Website Know What I Want?". American Mathematical Society. Retrieved October 31, 2016.
  46. ^ Macedo AA, Pollettini JT, Baranauskas JA, Chaves JC (2016). "A Health Surveillance Software Framework to deliver information on preventive healthcare strategies". J Biomed Inform. 62: 159–70. doi:10.1016/j.jbi.2016.06.002. PMID 27318270.CS1 maint: Multiple names: authors list (link)
  47. ^ Fernandez-Luque L, Karlsen R, Vognild LK (2009). "Challenges and opportunities of using recommender systems for personalized health education". Stud Health Technol Inform. 150: 903–7. PMID 19745443.CS1 maint: Multiple names: authors list (link)
  48. ^ Gomez-Uribe, Carlos A.; Hunt, Neil (28 December 2015). "The Netflix Recommender System". ACM Transactions on Management Information Systems. 6 (4): 1–19. doi:10.1145/2843948.
  49. ^ Rinke Hoekstra, The Knowledge Reengineering Bottleneck, Semantic Web – Interoperability, Usability, Applicability 1 (2010) 1, IOS Press
  50. ^ Dietmar Jannach; Markus Zanker; Alexander Felfernig; Gerhard Friedrich (2010). Recommender Systems:An Introduction. CUP. ISBN 978-0-521-49336-9. Archived from the original on 2015-08-31.
  51. ^ a b c Robin Burke, Hybrid Web Recommender Systems Archived 2014-09-12 at the Wayback Machine, pp. 377-408, The Adaptive Web, Peter Brusilovsky, Alfred Kobsa, Wolfgang Nejdl (Ed.), Lecture Notes in Computer Science, Springer-Verlag, Berlin, Germany, Lecture Notes in Computer Science, Vol. 4321, May 2007, 978-3-540-72078-2.
  52. ^ Alexander Felfernig and Robin Burke. Constraint-based Recommender Systems: Technologies and Research Issues, Proceedings of the ACM International Conference on Electronic Commerce (ICEC'08), Innsbruck, Austria, Aug. 19-22, pp. 17-26, 2008.
  53. ^ Ziegler, C.N., McNee, S.M., Konstan, J.A. and Lausen, G. (2005). "Improving recommendation lists through topic diversification". Proceedings of the 14th international conference on World Wide Web. pp. 22–32.CS1 maint: Multiple names: authors list (link)
  54. ^ Joeran Beel; Stefan Langer; Marcel Genzmehr; Andreas Nürnberger (September 2013). "Persistence in Recommender Systems: Giving the Same Recommendations to the Same Users Multiple Times" (PDF). In Trond Aalberg; Milena Dobreva; Christos Papatheodorou; Giannis Tsakonas; Charles Farrugia. Proceedings of the 17th International Conference on Theory and Practice of Digital Libraries (TPDL 2013). Lecture Notes of Computer Science (LNCS). 8092. Springer. pp. 390–394. Retrieved 1 November 2013.
  55. ^ Cosley, D., Lam, S.K., Albert, I., Konstan, J.A., Riedl, {J}. (2003). "Is seeing believing?: how recommender system interfaces affect users' opinions". Proceedings of the SIGCHI conference on Human factors in computing systems. pp. 585–592.CS1 maint: Multiple names: authors list (link)
  56. ^ {P}u, {P}., {C}hen, {L}., {H}u, {R}. (2012). "Evaluating recommender systems from the user's perspective: survey of the state of the art". User Modeling and User-Adapted Interaction: 1–39.CS1 maint: Multiple names: authors list (link)
  57. ^ Rise of the Netflix Hackers Archived January 24, 2012, at the Wayback Machine
  58. ^ "Netflix Spilled Your Brokeback Mountain Secret, Lawsuit Claims". WIRED. 17 December 2009. Retrieved 1 June 2015.
  59. ^ a b "Netflix Prize Update". Netflix Prize Forum. 2010-03-12.
  60. ^ Naren Ramakrishnan; Benjamin J. Keller; Batul J. Mirza; Ananth Y. Grama; George Karypis (2001). Privacy Risks in Recommender Systems. IEEE Internet Computing. 5. Piscataway, NJ: IEEE Educational Activities Department. pp. 54–62. CiteSeerX 10.1.1.2.2932. doi:10.1109/4236.968832. ISBN 978-1-58113-561-9.
  61. ^ Joeran Beel; Stefan Langer; Andreas Nürnberger; Marcel Genzmehr (September 2013). "The Impact of Demographics (Age and Gender) and Other User Characteristics on Evaluating Recommender Systems" (PDF). In Trond Aalberg; Milena Dobreva; Christos Papatheodorou; Giannis Tsakonas; Charles Farrugia. Proceedings of the 17th International Conference on Theory and Practice of Digital Libraries (TPDL 2013). Springer. pp. 400–404. Retrieved 1 November 2013.
  62. ^ {K}onstan, {J}.{A}., {R}iedl, {J}. (2012). "Recommender systems: from algorithms to user experience". User Modeling and User-Adapted Interaction: 1–23.CS1 maint: Multiple names: authors list (link)
  63. ^ {R}icci, {F}., {R}okach, {L}., {S}hapira, {B}., {K}antor {B}. {P}. (2011). "Recommender systems handbook". Recommender Systems Handbook: 1–35.CS1 maint: Multiple names: authors list (link)
  64. ^ Montaner, Miquel, L{\'o}pez, Beatriz, de la Rosa, Josep Llu{\'\i}s (2002). "Developing trust in recommender agents". Proceedings of the first international joint conference on Autonomous agents and multiagent systems: part 1. pp. 304–305.CS1 maint: Multiple names: authors list (link)
  65. ^ Beel, Joeran, Langer, Stefan, Genzmehr, Marcel (September 2013). "Sponsored vs. Organic (Research Paper) Recommendations and the Impact of Labeling" (PDF). In Trond Aalberg; Milena Dobreva; Christos Papatheodorou; Giannis Tsakonas; Charles Farrugia. Proceedings of the 17th International Conference on Theory and Practice of Digital Libraries (TPDL 2013). pp. 395–399. Retrieved 2 December 2013.CS1 maint: Multiple names: authors list (link)
  66. ^ a b Yong Ge; Hui Xiong; Alexander Tuzhilin; Keli Xiao; Marco Gruteser; Michael J. Pazzani (2010). An Energy-Efficient Mobile Recommender System (PDF). Proceedings of the 16th ACM SIGKDD Int'l Conf. on Knowledge Discovery and Data Mining. New York City, New York: ACM. pp. 899–908. Retrieved 2011-11-17.
  67. ^ Bouneffouf, Djallel (2012), "Following the User's Interests in Mobile Context-Aware Recommender Systems: The Hybrid-e-greedy Algorithm", Proceedings of the 2012 26th International Conference on Advanced Information Networking and Applications Workshops (PDF), Lecture Notes in Computer Science, IEEE Computer Society, pp. 657–662, ISBN 978-0-7695-4652-0[dead link]
  68. ^ Tuukka Ruotsalo; Krister Haav; Antony Stoyanov; Sylvain Roche; Elena Fani; Romina Deliai; Eetu Mäkelä; Tomi Kauppinen; Eero Hyvönen (2013). "SMARTMUSEUM: A Mobile Recommender System for the Web of Data". Web Semantics: Science, Services and Agents on the World Wide Web. 20: 657–662. CiteSeerX 10.1.1.676.7109. doi:10.1016/j.websem.2013.03.001.
  69. ^ Pimenidis, Elias; Polatidis, Nikolaos; Mouratidis, Haralambos (3 August 2018). "Mobile recommender systems: Identifying the major concepts". Journal of Information Science: 016555151879221. arXiv:1805.02276. doi:10.1177/0165551518792213.
  70. ^ a b Bouneffouf, Djallel (2013), DRARS, A Dynamic Risk-Aware Recommender System (Ph.D.), Institut National des Télécommunications
  71. ^ a b Lohr, Steve. "A $1 Million Research Bargain for Netflix, and Maybe a Model for Others". The New York Times.
  72. ^ R. Bell; Y. Koren; C. Volinsky (2007). "The BellKor solution to the Netflix Prize" (PDF).
  73. ^ Bodoky, Thomas (2009-08-06). "Mátrixfaktorizáció one million dollars". Index.
  74. ^ Lathia, N., Hailes, S., Capra, L., Amatriain, X.: Temporal diversity in recommender systems. In: Proceedings of the 33rd International ACMSIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010, pp. 210–217. ACM, New York
  75. ^ Turpin, Andrew H, Hersh, William (2001). "Why batch and user evaluations do not give the same results". Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval. pp. 225–231.CS1 maint: Multiple names: authors list (link)
  76. ^ Beel, Joeran; Genzmehr, Marcel; Langer, Stefan; Nürnberger, Andreas; Gipp, Bela (2013-01-01). A Comparative Analysis of Offline and Online Evaluations and Discussion of Research Paper Recommender System Evaluation. Proceedings of the International Workshop on Reproducibility and Replication in Recommender Systems Evaluation. RepSys '13. New York, NY, USA: ACM. pp. 7–14. CiteSeerX 10.1.1.1031.973. doi:10.1145/2532508.2532511. ISBN 9781450324656.
  77. ^ Lakiotaki, K.; Matsatsinis; Tsoukias, A (March 2011). "Multicriteria User Modeling in Recommender Systems". IEEE Intelligent Systems. 26 (2): 64–76. CiteSeerX 10.1.1.476.6726. doi:10.1109/mis.2011.33.
  78. ^ Gediminas Adomavicius, Nikos Manouselis, YoungOk Kwon. "Multi-Criteria Recommender Systems" (PDF). Archived from the original (PDF) on 2014-06-30.CS1 maint: Uses authors parameter (link)
  79. ^ "MovieLens dataset". 2013-09-06.
  80. ^ Jannach, Dietmar; Lerche, Lukas; Gedikli, Fatih; Bonnin, Geoffray (2013-06-10). Carberry, Sandra; Weibelzahl, Stephan; Micarelli, Alessandro; Semeraro, Giovanni, eds. User Modeling, Adaptation, and Personalization. Lecture Notes in Computer Science. Springer Berlin Heidelberg. pp. 25–37. CiteSeerX 10.1.1.465.96. doi:10.1007/978-3-642-38844-6_3. ISBN 9783642388439.
  81. ^ a b Turpin, Andrew H.; Hersh, William (2001-01-01). Why Batch and User Evaluations Do Not Give the Same Results. Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR '01. New York, NY, USA: ACM. pp. 225–231. CiteSeerX 10.1.1.165.5800. doi:10.1145/383952.383992. ISBN 978-1581133318.
  82. ^ Langer, Stefan (2015-09-14). "A Comparison of Offline Evaluations, Online Evaluations, and User Studies in the Context of Research-Paper Recommender Systems". In Kapidakis, Sarantos; Mazurek, Cezary; Werla, Marcin. Research and Advanced Technology for Digital Libraries. Lecture Notes in Computer Science. 9316. Springer International Publishing. pp. 153–168. doi:10.1007/978-3-319-24592-8_12. ISBN 9783319245911.
  83. ^ Basaran, Daniel; Ntoutsi, Eirini; Zimek, Arthur (2017). Proceedings of the 2017 SIAM International Conference on Data Mining. pp. 390–398. doi:10.1137/1.9781611974973.44. ISBN 978-1-61197-497-3.
  84. ^ Ekstrand, Michael D.; Ludwig, Michael; Konstan, Joseph A.; Riedl, John T. (2011-01-01). Rethinking the Recommender Research Ecosystem: Reproducibility, Openness, and LensKit. Proceedings of the Fifth ACM Conference on Recommender Systems. RecSys '11. New York, NY, USA: ACM. pp. 133–140. doi:10.1145/2043932.2043958. ISBN 9781450306836.
  85. ^ Konstan, Joseph A.; Adomavicius, Gediminas (2013-01-01). Toward Identification and Adoption of Best Practices in Algorithmic Recommender Systems Research. Proceedings of the International Workshop on Reproducibility and Replication in Recommender Systems Evaluation. RepSys '13. New York, NY, USA: ACM. pp. 23–28. doi:10.1145/2532508.2532513. ISBN 9781450324656.
  86. ^ a b Breitinger, Corinna; Langer, Stefan; Lommatzsch, Andreas; Gipp, Bela (2016-03-12). "Towards reproducibility in recommender-systems research". User Modeling and User-Adapted Interaction. 26 (1): 69–101. doi:10.1007/s11257-016-9174-x. ISSN 0924-1868.
  87. ^ Said, Alan; Bellogín, Alejandro (2014-10-01). Comparative recommender system evaluation: benchmarking recommendation frameworks. Proceedings of the 8th ACM Conference on Recommender Systems. RecSys '14. New York, NY, USA: ACM. pp. 129–136. doi:10.1145/2645710.2645746. hdl:10486/665450. ISBN 9781450326681.
  88. ^ Adamopoulos, Panagiotis; Bellogín, Alejandro; Castells, Pablo; Cremonesi, Paolo; Steck, Harald (2014-01-01). REDD 2014 – International Workshop on Recommender Systems Evaluation: Dimensions and Design. Proceedings of the 8th ACM Conference on Recommender Systems. RecSys '14. New York, NY, USA: ACM. pp. 393–394. doi:10.1145/2645710.2645780. ISBN 9781450326681.
  89. ^ "What You Need To Know About Anticipatory Design". Smashing Magazine. 2015-09-10. Retrieved 2017-12-06.

Further reading[edit]

Books

Kim Falk (January 2019), Practical Recommender Systems, Manning Publications, ISBN: 9781617292705

Scientific articles

External links[edit]

Retrieved from "https://en.wikipedia.org/w/index.php?title=Recommender_system&oldid=882692428"