Details

Autor(en) / Beteiligte

• Islam, Md. Rabiul
• Nawa, Shun
• Vargo, Andrew
• Iwata, Motoi
• Matsubara, Masaki
• Morishima, Atsuyuki
• Kise, Koichi

Titel

Quality Assessment of Crowdwork via Eye Gaze: Towards Adaptive Personalized Crowdsourcing

Ist Teil von

• Human-Computer Interaction – INTERACT 2021, 2021, Vol.LNCS-12933 (Part II), p.104-113

Ort / Verlag

Cham: Springer International Publishing

Erscheinungsjahr

2021

Link zum Volltext

Quelle

Alma/SFX Local Collection

Beschreibungen/Notizen

• A significant challenge for creating efficient and fair crowdsourcing platforms is in rapid assessment of the quality of crowdwork. If a crowdworker lacks the skill, motivation, or understanding to provide adequate quality task completion, this reduces the efficacy of a platform. While this would seem like only a problem for task providers, the reality is that the burden of this problem is increasingly leveraged on crowdworkers. For example, task providers may not pay crowdworkers for their work after the evaluation of the task results has been completed. In this paper, we propose methods for quickly evaluating the quality of crowdwork using eye gaze information by estimating the correct answer rate. We find that the method with features generated by self-supervised learning (SSL) provides the most efficient result with a mean absolute error of 0.09. The results exhibit the potential of using eye gaze information to facilitate adaptive personalized crowdsourcing platforms.