Scribd
Upload
13 views6 pages

lessons_from_amt_and_turkopticon_summary

This dissertation argues that online labor markets, such as Amazon Mechanical Turk, should address workers' concerns more substantively to ensure their sustainable livelihoods and the markets' long-term viability. It emphasizes the need to recognize workers as first-class stakeholders and to incorporate their input into market design and management practices. The document advocates for an interdisciplinary research agenda that connects software practice, empirical research, and value-rational analysis to better understand and improve the socio-economic impact of these labor markets.

Uploaded by

Ayotunde Oladoja
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
13 views6 pages

lessons_from_amt_and_turkopticon_summary

This dissertation argues that online labor markets, such as Amazon Mechanical Turk, should address workers' concerns more substantively to ensure their sustainable livelihoods and the markets' long-term viability. It emphasizes the need to recognize workers as first-class stakeholders and to incorporate their input into market design and management practices. The document advocates for an interdisciplinary research agenda that connects software practice, empirical research, and value-rational analysis to better understand and improve the socio-economic impact of these labor markets.

Uploaded by

Ayotunde Oladoja
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Human-centered computing

and the future of work

Lessons from Mechanical Turk and


Turkopticon, 2008–2015

M. Six Silberman, University of California, Irvine

This document is a summary of a dissertation.


The dissertation is available online at:

http://wtf.tw/diss.pdf

Contents of this document

Context 2
Main messages of the dissertation 3
Central argument Workers' concerns should be more 3
substantively and systematically addressed in the design
and operation of online labor markets.
Additional arguments
• In online labor markets, some workers are casual or 3
transient, while others are professionals.
• Workers who rely on income earned through online 3
labor markets to meet basic needs should be
considered first-class stakeholders.
• Workers in online labor markets are not usually the 3
narrowly self-interested profit maximizers of
classical economic theory.
• Online labor markets are not monolithic, perfectly 4
competitive markets; rather, each market is part of a
polycentric economic system composed of
complexly interlinked action situations.
• Institutions funding human-centered computing 4
(HCC) research should support an interdisciplinary
practice-oriented research agenda to understand the
consequences of current online labor market
designs and management practices, and to develop
new designs and practices that incorporate workers
who rely on market income as central stakeholders.
What the dissertation is 6
Context

Online labor markets such as Amazon Mechanical Turk (AMT),


Uber, and TaskRabbit are contributing to rapid changes in the
nature of work for hundreds of thousands of workers. These
markets stand to create significant new economic opportunities,
but current market designs and management practices typically
treat workers as second-class citizens. Take-home pay is typically
low compared to similar work in traditional employment
arrangements. Workers have limited means of influencing market
design or management practice, and therefore little control over
their own work arrangements. Low pay and limited control over
work arrangements make it hard for workers to create reliable,
sustainable livelihoods from the many uncoordinated work
opportunities online labor markets present. But reliable,
sustainable livelihoods matter for three reasons. First, workers
value them—even as they value the unique flexibility online labor
markets offer. Second, if workers cannot build reliable, sustainable
livelihoods, the labor pool reached by online labor markets will be
limited to casual or temporary workers—limiting the markets' own
long-term sustainability. Third, reliable, sustainable livelihoods are
crucial to socioeconomic mobility, a central concern of economic
policy.
This dissertation uses Amazon Mechanical Turk (AMT), an
online labor market for small information tasks—i.e., a "crowd
work market"—as a case through which to examine the
consequences of treating workers as second-class citizens, to argue
for future platform designs and management practices that treat
workers as central stakeholders, and to develop theory and method
for doing so.

2
Main messages of the dissertation

The central argument of this dissertation is that workers' concerns


should be more substantively and systematically addressed in
the design and operation of online labor markets.

Taking workers' concerns seriously will better allow them to create


reliable, sustainable livelihoods from the work opportunities these
new markets make available. It is therefore a crucial step in
ensuring the markets' own sustainability, and will contribute to
socioeconomic mobility in the economy at large.

Five messages elaborate this central argument. First, in online


labor markets, some workers are casual or transient, while
others are professionals, providing significant and reliable value
to customers on an ongoing basis and relying on income earned in
the market to meet their own basic needs. In AMT, the relatively
small fraction of "Turkers" who rely on Turking income to meet
basic needs do most of the work posted to the market. Most of
these professional Turkers are educated and highly skilled and live
in the United States. And many report that they work on AMT not
by choice but because they are unable to secure other employment.
Thus some of the narratives offered by researchers and employers
to justify low pay—e.g., that most workers who rely on Turking
income live in "developing" countries with low costs of living; that
most "developed"-country workers work mainly to pass time; that
crowd work is easy and workers relatively unskilled and
uneducated; and that workers freely choose to participate in AMT
and can easily choose other work if they find the pay too low—are
inaccurate.

Second, workers who rely on income earned through online labor


markets to meet basic needs should be considered first-class
stakeholders, alongside customers and shareholders. These
workers are strongly invested in the sustainability of the market.
When the market is designed appropriately, these workers can be
relied on to adhere to, and even enforce, market norms that benefit
all participants. Their concerns and input regarding the design and
operation of the market should be taken seriously. Formal
processes for eliciting their input should be developed and
integrated into market design and management practice.

Third, workers in online labor markets are not usually the


narrowly self-interested profit maximizers of classical economic
theory. Professionals—those workers who rely on income earned

3
through participation in the market—especially want market
transactions to produce good outcomes for everyone, and want the
market to be sustainable. They take professional pride in doing
good work and helping other market participants. They adhere to
norms they think will produce good outcomes for everyone, and
spend unpaid time discussing what those norms should be. Thus
researchers and platform operators should not see workers as
narrowly self-interested profit maximizers. A more realistic
approach is to see them as "situatedly rational" actors. "Situated
rationality" augments the notion of "bounded rationality"—i.e., that
actors have incomplete information and limited cognitive
capacities—with the observation that actors' actions, and even their
preferences, are shaped by a diversity of factors typically omitted
in classical economic analysis, including rules, norms, and
expectations. Certain market designs may induce situatedly
rational actors to act as narrowly self-interested profit maximizers,
but this result is not inevitable. On the contrary, it usually
produces suboptimal outcomes and may indicate poor market
design. Further, situatedly rational actors have mental models of
how the market works. These models are often sophisticated, but
rarely complete or perfectly accurate. And these models influence
market outcomes.

Fourth, online labor markets are not monolithic, perfectly


competitive markets. Nor is there a perfectly competitive "market
of markets." Rather, each market is part of a polycentric
economic system composed of complexly interlinked action
situations. This system is characterized by imperfect competition
and incomplete information. The notion of polycentricity indicates
that there are multiple decision making locations within the system
that, while formally independent, are interlinked by the
consequences of decisions taken at each location. Each decision
taken by an actor can be seen as occurring within an "action
situation." Action situations can be described by the characteristics
of the actors involved, their roles, the rules governing their actions,
the information available to them, the possible outcomes of their
actions, the relations between actions and outcomes, and actors'
valuations of outcomes. In a given action situation, an actor may
act according to a variety of logics—e.g., financial, institutional, or
moral. These logics may not be commensurable with one another,
and may not be formalizable.

Fifth and finally, institutions funding human-centered computing


(HCC) research should support an interdisciplinary practice-
oriented research agenda to understand the consequences of

4
current online labor market designs and management practices,
and to develop new designs and practices that incorporate
workers who rely on market income as central stakeholders.
This agenda should integrate software practice, empirical research,
theory development, and value-rational analysis. Software practice
and empirical research are familiar in HCC. Current HCC theories
must be expanded to larger scales of analysis and design. And the
three aforementioned well-established research modes should be
linked to value-rational analysis—the rigorous and broad-based
consideration of questions such as Where are we going in
computationally mediated work? Who gains and who loses? Is this
desirable? What should be done? The influence of designers' and
operators' understandings of such "nontechnical" issues in system
design and use has long been acknowledged in HCC research. But
online labor markets so tightly interweave the technical and the
ostensibly nontechnical that questions once considered
nontechnical can no longer be "outsourced" to social scientists or
regulators. The computational mediation of work calls for
computing researchers to take a more active role in the collective
process of understanding the social consequences of technology
design, articulating possible futures, distinguishing between what
is desirable and what is merely possible, making plain the
distribution of benefits and risks, and taking concrete steps to
create the institutional conditions required to develop systems and
practices that benefit a broad variety of stakeholders.

Industry collaboration will be crucial for the long-term


sustainability of such an effort. But such collaboration will be
complicated by the distinct institutional accountabilities and
cultures of research and business. Indeed business broadly is
grappling with parallel challenges, as can be seen in the relatively
new discourses on corporate social responsibility and social
entrepreneurship, the emergence of new structures for corporate
governance such as the B Corporation, and the development of new
computationally-mediated strategies for raising capital such as
crowdfunding. The greatest potential for creating broad-based
social value in online labor markets lies at the intersection of
expanded HCC theory and method and new organizational models
that aim to create sustainable value for a broader diversity of
stakeholders than traditional models.

5
What the dissertation is

The dissertation is an effort to link software practice, empirical


research, theory development, and value-rational analysis in the
context of online labor markets. It presents an empirical account of
the polycentric economic system around AMT (Chapter 2). This
account is written from my perspective as a builder and
administrator of Turkopticon, a system used by many professional
AMT workers to review employers. The dissertation then
introduces theoretical material from empirical social science
(Chapter 3) to structure this account. The perspective on the
empirical material offered by the social scientific theory informs
value-rational analysis, which in turn motivates a series of
proposals for future computing research and software practice
(Chapter 4).

You might also like