Evaluation of context-aware recommendation systems for information re-finding

Evaluation of context-aware recommendation systems for information re-finding

Sappelli, M.
Verberne, S.
Kraaij, W.

2016

In this article we evaluate context-aware recommendation systems for information re-finding by knowledge workers. We identify 4 criteria that are relevant for evaluating the quality of knowledge worker support: context relevance, document relevance, prediction of user action, and diversity of the suggestions. We compare 3 different context-aware recommendation methods for information re-finding in a writing support task. The first method uses contextual prefiltering and content-based recommendation (CBR), the second uses the just-in-time information retrieval paradigm (JITIR), and the third is a novel network-based recommendation system where context is part of the recommendation model (CIA). We found that each method has its own strengths: CBR is strong at context relevance, JITIR captures document relevance well, and CIA achieves the best result at predicting user action. Weaknesses include that CBR depends on a manual source to determine the context and in JITIR the context query can fail when the textual content is not sufficient. We conclude that to truly support a knowledge worker, all 4 evaluation criteria are important. In light of that conclusion, we argue that the network-based approach the CIA offers has the highest robustness and flexibility for context-aware information recommendation.

2016 ICT
DSC - Data Science
TS - Technical Sciences

http://resolver.tudelft.nl/uuid:c7f59e09-a3b1-4208-8e63-1355e21b76c2

https://doi.org/10.1002/asi.23717

745679

Journal of the Association for Information Science and Technology

article