Passed Stanford HCI quals. Received M.Sc. of Computer Science
Internship @Microsoft Research
Stanford Graduate Fellowship 2014-2017
Stanford School of Engineering Fellowship, 2013-2014
Distributed, parallel crowd workers can accomplish simple tasks through workflows. However, as shown in the process knowledge spectrum below, teams of collaborating crowd workers are necessary for complex goals. Unfortunately, a fundamental condition for effective teams — familiarity with other members — stands in contrast to crowd work’s flexible, on-demand nature.
We enable effective crowd teams with Huddler, a system for workers to assemble familiar teams even under unpredictable availability and strict time constraints. Huddler utilizes a dynamic programming algorithm to optimize for highly familiar teammates when individual availability is unknown. We first present a field experiment that demonstrates the value of familiarity for crowd teams: familiar crowd teams doubled the performance of ad-hoc (unfamiliar) teams on a collaborative task. We then report a two-week field deployment wherein Huddler enabled crowd workers to convene highly familiar teams in 18 minutes on average. This research advances the goal of supporting long-term, team-based collaborations without sacrificing the flexibility of crowd work.
Crowd work is typically limited to simple, context-free tasks because they are easy to describe and understand. In contrast, complex tasks require communication between the requester and workers to achieve mutual understanding, which can be more work than it is worth. We explore the notion of communication using structured microtasks in the domain of writing. Using an iterative design process we designed the following 5 mechanisms for communicating context:
Our studies compare these mechanisms with respect to the costs to the requester in providing information and the value of that information to workers while performing the task. We find that different mechanisms are effective at different stages of writing. For early drafts, asking the requester to state the biggest problem in the current write-up is valuable and low cost, while later it is more useful for the worker if the requester highlights the text that needs to be improved. These findings can be used to enable richer, more interactive crowd work than what currently seems possible. We incorporate the findings in a workflow for crowdsourcing written content using appropriately timed mechanisms for communicating with the crowd.
Dynamo is a platform to support the Mechanical Turk community in forming publics around issues and then mobilizing. We are researching new approaches, systems, and labor mechanisms for collective action online.
"Despite their variety, Turkers have something in common—a lack of power. They operate in a realm largely untouched by legislation, unions, and guilds. As a result, the inexperienced can find themselves earning well below minimum wage, or abused by underhanded employers. But a project out of Stanford University [and UC San Diego] is hoping to grant Turkers agency—and might begin to revolutionize the industry. Dynamo is a platform that gives Turkers a collective voice and, consequently, the chance to drive change." -The Daily Beast
The Vanishing 9-to-5: Ruthless and Liberating, YES! Magazine, 9/3/2016
On Demand, and Demanding Their Rights, The American Prospect magazine, 6/28/2016
Intellectual Piecework, The Chronicle, 2/16/2015
Amazon's Mechanical Turk workers protest: 'I am a human being, not an algorithm', The Guardian, 12/3/2014
Amazon’s Turkers Kick Off the First Crowdsourced Labor Guild, The Daily Beast, 12/3/2014
Amazon's Mechanical Turk workers want to be treated like humans, Engadget, 12/3/2014
Workers of Amazon services published open letters to Bezos saying "We are not an algorithm", Gigazine, 12/4/2014 ( Japanese)
Amazon Mechanical Turk workers begin letter-writing campaign, Crowdsourcing.org, 12/5/2014
Amazon Mechanical Turk: Artificial Artificial Intelligence, Rhizome Today, 12/5/2014
The proletariat web accesses the class consciousness and launches its first collective action to improve working conditions, Slate, 12/4/2014 (French)
Mechanical Turk workers protest, Xakep, 12/8/2014 (Russian)
Plenty of academic research passes through AMT or is about Turkers, but ethics boards (IRBs) who review and approve research protocols often don't know how workers want to be treated. Turkers have collectively authored these guidelines to help educate researchers and let Turkers hold them accountable to a higher standard.
Email us at email@example.com to add your support.
All around the world, people are writing letters about themselves to a distant, all-powerful figure who can make their dreams come true. But this year, many of those letters will be addressed to Amazon boss Jeff Bezos rather than Father Christmas. - The GuardianThe goals of this campaign are to publicly state that:
How does the relationship between users and their personal collection of social media data shift over time as this “digital trace” is formed? We explored these relationships, tensions, and management strategies involved. User culture of social networks has long been considered as “right here, right now”; However, we believe Facebook Timeline -explicitly supporting access to past data- has created a unique opportunity to rethink the nature of social media, the information it affords, and the user culture around it. We conducted a user study and discussed models and design ideas to address what we found.
Our study shows that people experience the Facebook platform as consisting of three different functional regions: a performance region for managing recent data and impression management, an exhibition region for longer term presentation of self-image, and a personal region for archiving meaningful facets of life. Further, users' need for presenting and archiving data in these three regions is mediated by temporality. These findings trigger a discussion of how to design social media that support these dynamic and sometimes conflicting needs.