Polls Can Create an Illusion of Public Opinion

George Bishop is an independent survey research consultant, a retired professor of political science at
the University of Cincinnati and the author of The Illusion of Public Opinion.

UPDATED NOVEMBER 30, 2015, 3:21 AM

George Gallup thought he could measure the Will of the People. Like every other pollster since then, he believed
he could not only accurately predict the outcome of elections with his new statistical-survey tool, within a margin
of sampling error; but he could also measure public opinion on social, economic and political issues with the same
precision. It was just a matter of applying the same methodology, plus careful attention to how the questions were
asked.

With some notable exceptions, polls taken just prior to presidential elections have
proved to be quite accurate. Fortunately, we can validate the accuracy of those By asking questions
predictions on Election Day. that are vaguely
But that’s not the case when pollsters ask survey respondents about policy issues worded about topics
such as how President Obama or the Congress is dealing with “the economy” and that the average
“the federal budget deficit” or “foreign affairs.” Here pollsters often create an illusion citizen poorly
of public opinion by asking respondents questions that are vaguely worded about understands,
topics on which the average citizen is poorly informed. Even worse, respondents are pollsters create
often asked to answer questions they’re psychologically incapable of answering, such
subjective reality.
as the “reasons” why they prefer a given candidate or why they favor this or that
policy. Cognitive neuroscientists tell us that we simply do not have introspective
access to the unconscious processes that drive our opinions and preferences. We’re clueless.

Take, for example, the Gallup question on presidential approval. “Do you approve or disapprove of the way Barack
Obama is handling his job as president?” What does “handling his job as president” mean to respondents? The
same goes for the “Affordable Care Act.” Do many respondents know much about it other than that it’s
“Obamacare” and that they’re for or against him? Do the meanings of questions vary across respondents and over
time? If so, it violates a cardinal assumption of survey practice: that a survey question should mean the same thing
to all respondents. Otherwise we’re comparing apples with oranges.

If we ask respondents “why” they approve or disapprove of how President Obama is handling this or that problem,
they really don’t know why. All they can do is come up with plausible justifications or “reasons” for their opinions
after-the-fact. Willing respondents will answer our questions if you ask them, however vague those questions
might be and however uninformed they might be — unless you give them a chance to admit they don’t know much
about the issue. If you build the questions, they will answer them. And that’s how pollsters -- unwittingly perhaps -
- manufacture the “will of the people.”

Join Opinion on Facebook and follow updates on twitter.com/roomfordebate.

Topics: Politics, democracy, polls

Polls Rarely Ask About Concerns Vital to
Minority Voters
Taeku Lee is a professor of political science and law at the University of California, Berkeley, and the director
of Asian American Decisions.

UPDATED NOVEMBER 30, 2015, 3:21 AM

The 2016 presidential campaign, still a youngster by any reasonable standard, is already more drenched in poll
numbers than ever before. The lure is obvious. From its founding moments in the 1930s, polls were vaunted for
their potential to deploy science in the service of democracy.

Yet polling today is a world apart from the salad days of George Gallup, Elmo Roper
and Archibald Crossley. These pioneers could not have anticipated the dramatic, The typical poll
wholesale transformation of the demoi since the 1960s. In the 1930s, the electorate interviews fewer
was, for most intents and purposes, white — especially in light of the ruthless
than 1 in 10 targeted
efficacy of Jim Crow laws in the South. Today, the country is roughly one-third
nonwhite and we are readying for a future (sometime mid-century) in which whites
respondents.
are no longer the majority. Minorities and those
without college
To be fair, diversity itself does not threaten the promise of polling. But polls serve degrees are
the aspirations of democracy qua public opinion best when they are both
underrepresented.
representative and relevant. That’s where the vexation arises.

On representativeness, the typical poll today successfully interviews fewer than 1 in 10 targeted respondents, with
racial minorities, noncitizens and persons without college degrees among the underrepresented. Moreover, in the
conduct of surveys nonwhites are often likelier to be left out of a sample altogether. Telephone surveys reliant on
landlines will miss those who use cell phones only; Internet surveys will miss those on the lower rungs of
the digital divide; both examples involve Latinos and African-Americans. Many polls still interview only in
English (and scarcely ever in Asian languages). Survey samples are generally exclusive of institutionalized
populations, a sobering fact in our era of mass incarceration.

Even if representativeness were achieved, polls properly mirror public opinion only when they ask about the
concerns that are actually on people’s minds, concerns about which held views are firm and compelled to find
voice. For a range of reasons – survey time is precious, parties and their candidates are incented to stay mum on
race, and so on – concerns vital to minority voters rarely make the cut.

But for a more deliberate and concerted effort to address these deficits of relevance and representativeness, polls
will continue to churn out red meat that feeds the public’s frenzy for the blood sport of electoral politics. In so
doing, however, polls will more closely approximate a carnival fun house mirror than the looking glass on society
that Gallup envisioned.

Join Opinion on Facebook and follow updates on twitter.com/roomfordebate.

Topics: Politics, democracy, polls

When Judges Felt Polls Gave Them the Power to Act
Lanae Erickson Hatalsky is the director of social policy and politics for Third Way.

UPDATED NOVEMBER 30, 2015, 3:21 AM

“Judges are like umpires. Umpires don't make the rules; they apply them. … It's my job to call balls and strikes.”
Chief Justice Roberts coined this maxim during his Senate confirmation hearings. It has now become a refrain for
judicial nominees anxious to be confirmed.

So why in some high-profile cases do they appear to be influenced by the crowd? Because in some cases, they
should be.

Take marriage. When the Supreme Court handed down a ruling this summer giving
gay couples the freedom to marry from coast to coast, critics lambasted the justices The Supreme Court
for being swayed by polls showing an increase in the numbers of Americans who legalized same-sex
supported it. They meant this as a criticism. But while the rules of baseball are static,
marriage
constitutional law is not.
nationwide after
When determining whether something is protected under the Constitution as a surveys showed that
fundamental right, judges must ask whether it is deeply embedded in the history and 60 percent of
traditions of our country. But those traditions evolve, as do the state laws that often Americans
reflect them.
supported it.
On marriage, when the Supreme Court jumped into the debate in late 2012 by
accepting challenges to the Defense of Marriage Act and California’s Proposition 8, voters in four states had just
given marriage advocates their first ever set of victories at the ballot box. The court ruled narrowly on those initial
cases. At the time, public support for marriage was growing but still hovered just over 50 percent. As Justice Ruth
Bader Ginsburg, a liberal lion of the court, telegraphed, the court’s supporters of marriage for gay couples worried
that approving it nationwide at that moment could provoke a decades-long backlash, as had happened after Roe v.
Wade legalized abortion across the country.

By the time the Supreme Court granted gay couples the right to marry nationwide this year, 60 percent of the
country supported it.

Ultimately, public opinion did affect the umpires, but unlike in baseball, that is exactly how our system is
supposed to work.

Join Opinion on Facebook and follow updates on twitter.com/roomfordebate.

Polls Remain Prominent Despite Years of Skepticism
Sarah Igo, an associate professor of history at Vanderbilt University, is the author of "The Averaged American:
Surveys, Citizens and the Making of a Mass Public."

UPDATED NOVEMBER 30, 2015, 3:21 AM

The historian James Bryce suggested in 1888 that public opinion ought to be “the real ruler of America,” but
lamented the lack of “machinery for weighing or measuring the popular will from week to week or month to
month.”

We now have that machinery, of course. Those who created modern polling in the 1930s, George Gallup among
them, shared Bryce’s dream. They believed that there was a method — shorn of power or politics — that could
determine what the public really wanted, a scientific solution to the dilemmas of democratic representation. A vast
industry grew up around that promise. Today it weighs and measures opinion at a pace that would have surely
astonished James Bryce.

But does it capture anything resembling the “popular will”? Or has the machinery
itself distorted or even suppressed it? The powerful technocratic strain in American Did biases creep into
culture produced its own counter-currents. Long before today’s epidemic of queries? Did polls
nonresponses and push polls, critics were poking holes in the notion that something
report on the public
as slippery as the popular will could be measured. If “public opinion” existed at all,
thinking, or
they were quite sure it would not manifest itself in the stable, clear and quantifiable
form the polls favored. manufacture it? Did
they benefit the
Nor, despite Gallup’s hopes, were issues of power and politics banished. As polls people, or special
were embedded in American life, so too were popular understanding of their flaws.
interests?
Who and what was (and wasn’t) asked? How did biases creep into pollsters’ queries
as well as the answers they tallied? Was it possible that the opinions being measured
weren’t real at all, but mere artifacts of the peculiar interaction between pollster and respondent? Did polls report
on the public agenda, or manufacture it? And whom did they benefit: the people, or an array of organized interests
that spoke in their name?

In this light, the skepticism with which Americans view the polls (a fact we know only because of opinion surveys,
of course) should be welcome. The real problem is not imperfectly tuned polls: it is that vigorous public criticism
has not substantially altered their prominence in U.S. public life. Polls may not be the fix for democracy that Bryce
and Gallup imagined. But the questions they raise are ones that democratic deliberation may yet fix.

Join Opinion on Facebook and follow updates on twitter.com/roomfordebate.

Topics: Politics, democracy, polls

Politicians Use Polls to Adjust Their Message
Diane J. Heith, professor and chair of government and politics at St. John’s University, is the author of "The
Presidential Road Show: Public Leadership in an Era of Party Polarization and Media Fragmentation," "Polling to
Govern: Public Opinion and Presidential Leadership" and is co-author of "Presidents and the American Presidency."

UPDATED NOVEMBER 30, 2015, 3:21 AM

As part of a permanent campaign, political leaders use polls to craft their language, track responses to their
speeches and identify who is for them, who against them, and why. Polling helps politicians maintain and expand
their coalition from one election to the next by tracking attitudes.

Politicians also benefit from polling in the same way companies marketing a product
do. It is easier to persuade when you understand your audience. The Reagan If polls are used to
administration discovered that calling a space-based defense program “Star Wars” find persuasive
appealed to the public more than when it was called the “Strategic Defense
language, our
Initiative.”
attitudes can suborn
The benefit for the public is less obvious. A representative democracy is based on support. And
input from the public in elections. Some citizens take the democratic mandate opinions of the
further and express their opinions by contacting their elected officials, or by holding inattentive add little
up signs on street corners, or by marching on Washington. In any of those scenarios,
to the policy-making
citizen preferences drive participation.
process.
Many citizens do not actively participate, even in elections. However, polls collect
the opinions of all citizens whether they are active, inactive, partisan, independent,
knowledgeable, disconnected or oblivious. While we want more participation, gathering the opinions of the
inattentive or unknowledgeable does not add value to the policy-making process.

The quality and use of polling data can also raise concerns. Most polls reveal simplistic preferences: “Yes, I agree;”
or “No, I don’t.” Answering a complex, multifaceted question requires the respondent to understand the question,
the choices and to truly have a preference.

If polls are used to identify persuasive language, then our attitudes can suborn support. Knowledge of poll data
can also persuade: learning that 55 percent of Americans believe something or support someone can produce a
bandwagon effect for those with limited knowledge. Plus, close poll results that fall within the margin of error
make positions on policy appear certain when they are actually uncertain.

Citizens want their views represented while decisions are being made. Polls provide that opportunity in a way that
elections do not. But politicians control how polls are used, and if used to manipulate, then we may be worse off
than if we didn’t have them at all.

Join Opinion on Facebook and follow updates on twitter.com/roomfordebate.

Topics: Politics, democracy, polls

Polls Can Give People a Stronger Voice
Arthur Lupia, the Hal R. Varian professor of political science at the University of Michigan, is the author of
"Uninformed: Why Voters Know So Little about Politics and What We Can Do About It." He is on Twitter.

UPDATED OCTOBER 12, 2016, 2:11 PM

Polls are a popular whipping-boy in politics. They are blamed for politicians’ bad decisions and the poor state of
democracy.

This, despite the fact that in most elections in America, which occur at the local level, there are no polls because
candidates can’t afford them or the locality is so small that you really don’t need a poll.

Even without polls, the same complaints emerge. Many people complain about their local politicians’ bad
decisions and the poor state of democracy.

People complain because politics entails conflicts that result in lots of people not
getting what they want (i.e., they lose elections or legislative votes). Many people Without polls,
who don’t get what they want need something to blame – so they blame the polls. politicians and
If you think that polls are destroying democracy, then think about an alternative. special interests
Without polls, politicians, special interests and certain members of the news media could make claims
would still have strong incentives to make forceful claims about the public’s views. about public opinion
Unconstrained by credible polling data, these individuals could spin incredible and spin tales about
stories about the great public support for their endeavors. the great support for
This, by the way, is not a hypothetical situation. It is exactly how claims about public their policies.
opinion are circulated and adjudicated in a range of democratic and authoritarian
regimes that lack credible polling organizations. Without good polls, it is difficult or impossible to confront bold
claims by powerful people. Ask Vladimir Putin.

Polls are imperfect. They always have been. It has always been challenging to draw accurate inferences about a
large population by interviewing only a small percentage of the population. The best pollsters understand this.
Professional polling organizations, such as the American Association of Public Opinion Researchers, work hard to
help polling organizations establish rigorous best practices and help individuals learn as much as possible from
polling data.

At the same time, many second-rate polling organizations try to pass themselves off as credible sources of
information despite their cut-rate methods and carelessness in reporting critical characteristics of their data (such
as response rates). If polling organizations are honest about what they have, and if media organizations do their
part and help their audiences differentiate reliable polls from unreliable polls, polls can strengthen democracy by
providing important information about what citizens want and think.

Join Opinion on Facebook and follow updates on twitter.com/roomfordebate.

Topics: Politics, democracy, polls