Modeling Crowdsourcing Systems: Design and Analysis of Incentive and Reputation Systems

Add to your list(s) Download to your calendar using vCal

• John C.S. Lui (The Chinese University of Hong Kong)
• Thursday 03 July 2014, 15:00-16:00
• MR14, Centre for Mathematical Sciences, Wilberforce Road, Cambridge.

If you have a question about this talk, please contact Felix Fischer.

Crowdsourcing systems like Yahoo!Answers, Amazon Mechanical Turk, and Google Helpouts, etc., have seen an increasing importance and prevalence in the past few years. Crowdsourcing system serves as an efficient platform for “requesters” to outsource “tasks” to a crowd of distributed “workers,” who in return solve the assigned tasks and reply to requesters with solution. The participation of users (requesters and workers), high quality solutions, and a fair rating system are critical to the revenue of a crowdsourcing system. To address these challenges, we develop simple models to characterize workers’ strategic behavior, which allows different levels of contribution. We design a class of simple but effective incentive mechanisms, which consist of a “task bundling scheme” and a “rating system”, and pay workers according to solution ratings from requesters. We also propose a probabilistic model to capture various human factors, e.g., bias, in rating, as well as quantify its impact on the incentive mechanism, which is shown to be highly robust. We characterize the design space of rating systems, and quantify the impact of rating systems and the bundling scheme on the incentive mechanism. We show that the design of a rating system is a fundamental tradeoff between “crowdsourcing system efficiency” (i.e., the number of tasks can be solved for the given rewards) and the “cost in expressing solution ratings” (i.e., the time or cognitive cost). We also study a class of “threshold based rating systems,” which allows to vary the minimum requirement on task solutions.

This talk is part of the Optimization and Incentives Seminar series.

This talk is included in these lists:

• All CMS events
• All Talks (aka the CURE list)
• CMS Events
• Cambridge talks
• DPMMS Lists
• DPMMS info aggregator
• DPMMS lists
• Economics and Computer Science Talks
• Hanchen DaDaDash
• Interested Talks
• MR14, Centre for Mathematical Sciences, Wilberforce Road, Cambridge
• Optimization and Incentives Seminar
• School of Physical Sciences
• Statistical Laboratory info aggregator
• Trust & Technology Initiative - interesting events
• bld31

Note that ex-directory lists are not shown.