Recommender Systems is one of our core competencies. We particularly focus on the development of real-world recommender systems, user modelling, the evaluation of recommender systems, and several aspects more (e.g. labelling of recommendations, recommendation persistence, position bias, choice overload, and recommender system datasets). For more details on our work on recommender systems read the blog posts below or have a look at the menu above. You find our main website at http://recommender-systems.ie and the current page at http://ireland.recommender-systems.ie.
Recently, we spoke with TechCentral.ie, Ireland’s best well-known and respected technology magazine, about recommender systems, personalisation, the challenges of the GDPR and our work in the field of recommender-systems as-a-service and machine learning. The full interview is available here http://www.techcentral.ie/focus-research-joeran-beel-adapt/
On 28th February, we released version 1.1.1 of Mr. DLib’s recommender system with some minor improvements and bug fixes: Improved 404 error handling for unknown document IDs Fix: The order of authors in the XML was not sorted properly Several internal changes (adjusted logging table; click time is not updated any Read more…
Six months ago, we launched Mr. DLib’s recommendations-as-a-service for Academia. Time, to look back and provide some numbers: Since September 2016, Mr. DLib´s recommender system has delivered 60,836,800 recommendations to our partner Sowiport, and Sowiport’s users have clicked 91,545 of the recommendations. This equals on overall click-through rate (CTR) of Read more…
We are proud to announce version 1.1 of Mr. DLib’s Recommender-System as-a-Service. The major new features are: A JavaScript Client to request recommendations from Mr. DLib. The JavaScript offers many advantages compared to a server-side processing of our recommendations. Among others, the main page will load faster while recommendations are requested in the Read more…
Currently, all of Docear’s online services are down, including the recommender system. This means, you cannot register, log-in to download backups, or receive recommendations. As we have no time right now for the development of Docear, we are afraid that we won’t be able to fix this problem anytime soon. Read more…
Our paper titled “Stereotype and Most-Popular Recommendations in the Digital Library Sowiport” is accepted for publication at the 15th International Symposium on Information Science (ISI) in Berlin. Abstract: Stereotype and most-popular recommendations are widely neglected in the research-paper recommender-system and digital-library community. In other domains such as movie recommendations and hotel Read more…
Two of our papers about weighting citations and terms in the context of user modeling and recommender systems got accepted at the iConference 2017. Here are the abstracts, and links to the pre-print versions: Evaluating the CC-IDF citation-weighting scheme: How effectively can ‘Inverse Document Frequency’ (IDF) be applied to references? In Read more…
A demonstration paper about the integration of Mr. DLib in JabRef is accepted for publication at ECIR 2017. We will update this post soon with more information and a pre-print.
Content-based filtering recommendations suffer from the problem that no human quality assessments are taken into account. This means a poorly written paper ppoor would be considered equally relevant for a given input paper pinput as high-quality paper pquality if pquality and ppoor contain the same words. We elevate for this problem by using Mendeley’s readership data Read more…
We are glad to announce that JabRef – one of the most popular open-source reference manager – will soon be using Mr. DLib´s recommender system to provide article recommendations to its users. The team of Mr. DLib and JabRef had several Skype calls, and we finally agreed about a concept how to integrate Read more…
Our servers for Mr. DLib’s recommender-system as-a-service (RaaS) are now monitored by UptimeRobot, a free monitoring service. You can access all our RaaS server statuses at this URL https://stats.uptimerobot.com/WLL5PUjN6 and you will see a dashboard like this: A click on one of the server names will show you more details, e.g. https://stats.uptimerobot.com/WLL5PUjN6/778037437
We have integrated several new recommendation algorithms into Mr. DLib. Some recommendation algorithms are only ought as baselines for our researchers, others hopefully will further increase the effectiveness of Mr. DLib. Overall, Mr. DLib now uses the following recommendation algorithms in its recommender system: Random Recommendations The approach recommendation randomly picks Read more…
So far, Mr. DLib’s recommender system was running only on a single server. Consequently, when me messed up something in the development environment, sometimes the production system was affected, i.e. down. From today on, we have two additional dedicated servers running, meaning we have a total of three recommender-system servers, one for Read more…
Update on 2018-03-15: We Are Hiring 1 Software Engineer & 1 Software Architect / Product Owner for a Recommender-System Business Start-up Updated on 2017-08-14: Here at Docear and Mr. DLib we have many exciting projects in the field of recommender systems, user modelling, personalisation, and adaptive systems (primarily with Read more…
We are proud to announce that the social science portal Sowiport is using Mr. DLib´s recommender-system as-a-service as first pilot partner. Sowiport pools and links quality information from domestic and international providers, making it available in one place. Sowiport currently contains 9.5 million references on publications and research projects. The Read more…
Update 2016-01-12: The salary in Tokyo would be around 1.600 US$ per month, not 1.400. 2015 has been a rather quiet year for Docear, but 2016 will be different. We have lots of ideas for new projects, and even better – we have funding to pay at least 1 Master or Read more…
One reason why we originally started the development of Docear was our interest in how people are creating mind-maps and how the information contained in mind-maps could be used for building recommender systems and other user-modeling applications. As a result, we developed Docear’s research-paper recommender system, and if you are interested in how Read more…
Our paper “The Architecture and Datasets of Docear’s Research Paper Recommender System” was accepted at the 3rd International Workshop on Mining Scientific Publications (WOSP 2014), which is held in conjunction with the ACM/IEEE Joint Conference on Digital Libraries (JCDL 2014). This means, we will be in London from September 9 Read more…
As some of you might know, I am a PhD student and the focus of my research lies on research-paper recommender systems. Now, I am about to finish an extensive literature review of more than 200 research articles on research paper recommender systems. My colleagues and I summarized the findings Read more…
We recently submitted a paper to UMAP (The Conference on User Modelling, Adaptation, and Personalization). The paper was about how mind-maps could be utilized by information retrieval applications such as recommender systems. The paper got accepted, which means we will be in Aalborg, Denmark from July 7 until July 11 to present the Read more…
We kindly ask you to participate in a brief study about Docear’s recommender system. Your participation will help us to improve the recommender system, and to secure long-term funding for the development of Docear in general! If you are willing to invest 15 minutes of your time, then please continue reading. Participate Read more…
Update: February 18, 2014: No bugs were reported, as such we declare Docear 1.03 with its recommender system as stable. It can be downloaded on the normal download page.
With Docear 1.0.3 beta we have improved PDF handling, the recommender system, provided some help for new users and enhanced the way how you can access your mind maps online.
PDF Handling
We fixed several minor bugs with regard to PDF handling. In previous versions of Docear, nested PDF bookmarks were imported twice when you drag & dropped a PDF file to the mind map. Renaming PDF files from within Docear changed the file links in your mind maps but did not change them in your BibTeX file. Both issues are fixed now. To rename a PDF file from within Docear you just have to right-click it in Docear’s workspace panel on the left hand side and it is important that the mind maps you have linked the file in, are opened. We know, this is still not ideal, and will improve this in future versions of Docear.
Rate Your Recommendations
You already know about our recommender system for academic literature. If you want to help us improving it, you can now rate how good a specific set of recommendations reflects your personal field of interest. Btw. it would be nice if you do not rate a set of recommendations negatively only because it contains some recommendations you received previously. Currently, we have no mechanism to detect duplicate recommendations.
Yesterday, we published a pre-print on the shortcomings of current research-paper recommender system evaluations. One of the findings was that results of offline and online experiments sometimes contradict each other. We did a more detailed analysis on this issue and wrote a new paper about it. More specifically, we conducted a comprehensive evaluation of a set of recommendation algorithms using (a) an offline evaluation and (b) an online evaluation. Results of the two evaluation methods were compared to determine whether and when results of the two methods contradicted each other. Subsequently, we discuss differences and validity of evaluation methods focusing on research paper recommender systems. The goal was to identify which of the evaluation methods were most authoritative, or, if some methods are unsuitable in general. By ‘authoritative’, we mean which evaluation method one should trust when results of different methods contradict each other.
Bibliographic data: Beel, J., Langer, S., Genzmehr, M., Gipp, B. and Nürnberger, A. 2013. A Comparative Analysis of Offline and Online Evaluations and Discussion of Research Paper Recommender System Evaluation. Proceedings of the Workshop on Reproducibility and Replication in Recommender Systems Evaluation (RepSys) at the ACM Recommender System Conference (RecSys) (2013), 7–14.
Our current results cast doubt on the meaningfulness of offline evaluations. We showed that offline evaluations could often not predict results of online experiments (measured by click-through rate – CTR) and we identified two possible reasons.
The first reason for the lacking predictive power of offline evaluations is the ignorance of human factors. These factors may strongly influence whether users are satisfied with recommendations, regardless of the recommendation’s relevance. We argue that it probably will never be possible to determine when and how influential human factors are in practice. Thus, it is impossible to determine when offline evaluations have predictive power and when they do not. Assuming that the only purpose of offline evaluations is to predict results in real-world settings, the plausible consequence is to abandon offline evaluations entirely.
As you might know, Docear has a recommender system for research papers, and we are putting a lot of effort in the improvement of the recommender system. Actually, the development of the recommender system is part of my PhD research. When I began my work on the recommender system, some years ago, I became quite frustrated because there were so many different approaches for recommending research papers, but I had no clue which one would be most promising for Docear. I read many many papers (far more than 100), and although there were many interesting ideas presented in the papers, the evaluations… well, most of them were poor. Consequently, I did just not know which approaches to use in Docear.
Meanwhile, we reviewed all these papers more carefully and analyzed how exactly authors conducted their evaluations. More precisely, we analyzed the papers for the following questions.
After three demo-papers were accepted for JCDL 2013, we just received notice that another three posters were accepted for presentation at TPDL 2013 on Malta in September 2013. They cover some novel aspects of recommender systems relating to re-showing recommendations multiple times, considering user demographics when evaluating recommender systems, and investigating the effect of labelling recommendations. However, you can read the papers yourself, as we publish them as pre-print:
Paper 1: The Impact of Users’ Demographics (Age and Gender) and other Characteristics on Evaluating Recommender Systems (Download PDF | Doc)
In this paper we show the importance of considering demographics and other user characteristics when evaluating (research paper) recommender systems. We analyzed 37,572 recommendations delivered to 1,028 users and found that elderly users clicked more often on recommendations than younger ones. For instance, users with an age between 20 and 24 achieved click-through rates (CTR) of 2.73% on average while CTR for users between 50 and 54 was 9.26%. Gender only had a marginal impact (CTR males 6.88%; females 6.67%) but other user characteristics such as whether a user was registered (CTR: 6.95%) or not (4.97%) had a strong impact. Due to the results we argue that future research articles on recommender systems should report demographic data to make results better comparable.