Details

Autor(en) / Beteiligte

• Mogenet, Adrien
• Pham, Tuan Anh Nguyen
• Kazama, Masahiro
• Kong, Jialin

Titel

Predicting online performance of job recommender systems with offline evaluation

Ist Teil von

• Proceedings of the 13th ACM Conference on Recommender Systems, 2019, p.477-480

Ort / Verlag

New York, NY, USA: ACM

Erscheinungsjahr

2019

Link zum Volltext

Quelle

ACM Digital Library

Beschreibungen/Notizen

• At Indeed, recommender systems are used to recommend jobs. In this context, implicit and explicit feedback signals we can collect are rare events, making the task of evaluation more complex. Online evaluation (A/B testing) is usually the most reliable way to measure the results from our experiments, but it is a slow process. In contrast, the offline evaluation process is faster, but it is critical to make it reliable as it informs our decision to roll out new improvements in production. In this paper, we review the comparative offline and online performances of three recommendations models, we describe the evaluation metrics we use and analyze how the offline performance metrics correlate with online metrics to understand how an offline evaluation process can be leveraged to inform the decisions.