Noto Bene: The duty of candor requires that I mention that the October 23, 2016, poll included 874 LIKELY voters. But it also included 517 NOT LIKELY TO VOTE participants.
Here's another example
of using oversampling to steer the outcome of a poll.
ABC News Washington Post Tracking Poll today
is being reported to show that Clinton has opened a 12 point lead over Trump.
Fascinating stuff. Odd, too, considering that less than two weeks ago, a
similar poll reported by ABC/WaPo showed Clinton leading by 6 points.
Let's call today's poll the "new
poll" and the other poll the "old poll."
The old poll was conducted by interviewing
fewer than 800 "LIKELY VOTERS."
The new poll was conducted by interviewing
just over 1300 subjects, of whom 874 were LIKELY VOTERS.
The results of the new poll, however, are not
the results of just those 874 likely voters. That poll interviewed 1391 people.
Nearly 500 people were interviewed and their responses included in the results
of the poll despite the fact that they were NOT LIKELY TO VOTE.
Say What?
Yes.
The total result of the poll included 874
folks that are likely to vote and FIVE HUNDRED SEVENTEEN respondents who ARE
NOT LIKELY TO VOTE.
NOT LIKELY TO VOTE.
Think about the meaning of that phrase and how
it connects to the matter with which we tend to ASSUME news organizational
polling is concerned. Maybe that is the place to start.
Why do you look at news reports of polling?
Is it a search for confirmation? Sort of an
internal dialog, "See, the majority of folks agree with me" kind of
thing?
Is it an effort to see into the future? Sort
of a "Well, let's see how this is going to turn out? kind of thing?
I think these are two of the most common
responses to expect when you get honest responses from folks as to why they are
interested in such polls.
But suppose what you are looking for in
polling is not what campaigns and/or news agencies are seeking?
You realize that "push polling"
exists. Those polls that, rather than gauging the temper of the people seek to
put fire under them or dump cold water on them.
Let's not jump to any conclusions yet. But,
suppose a polling agency and its partners wanted to steer the outcome of a
poll. If they did, which approach do you think might be more effective. I'll
give you two choices.
Choice A.
After confirming that the interview subject
fits poll requirements (likely voter or not; party affiliation or not; racial
and cultural demographic or not), ask one question:
"In a two way race between Hillary
Clinton or Donald Trump, for whom would you vote?"
Choice B.
Again, after confirming that the interview
subject fits poll requirements (likely voter or not; party affiliation or not;
racial and cultural demographic or not), ask three questions:
"First, Donald Trump says the election
may be rigged through election fraud. Do you think he is expressing a
legitimate concern or that he is making excuses for a possible unfavorable
outcome?"
"Second, Donald Trump refuses to say if
he will accept the outcome of the election. Do you approve or disapprove of his
remark?"
"Third, In a two way race between Hillary
Clinton or Donald Trump, for whom would you vote?"
Now, you make your own judgment. But for me,
asking a couple of good ground penetrating bomb blast questions might just be
the kind of thing that deforms outcomes that might otherwise be gotten by
simply asking for whom the likely voter is likely to vote.
But Choice A above is not what Langer
Associates did in its polling for ABC News and the Washington Post. Instead, it
took Choice B, peppering participants with those questions.
The ABC/WaPo poll also engaged in modeling.
This approach looks at some past election, determines participation percentages
by party and then skews its sampling according to those numbers.
Here, the Langer agency skewed its sample.
Here's how they described their skew: "Partisan divisions are 36-27-31
percent, Democrats-Republicans-independents." In other words, the agency
set itself a quota task: 36% of those interviewed must be Democrats, 27% must
be Republicans, 31% must be Independents. These kinds of partisan divisions are
included in most polls. The exact percentages change. For example, why 36-27-31
when the Pew Center says that, currently, the breakdown in partisanship is
33-29-34?
You can see, I'm sure, then, that the
specific, formulaic breakdown used in selecting polling participants can skew
the numbers, and it becomes important, if polls are something you have to deal
with or choose to deal with, to KNOW what the pollster is using for partisan
division, and to know WHY they are using any particular spread.
I'm going to wrap here by returning to that
517 number.
Remember, this poll was one conducted by
interviewing 517 people, according to the polling methodology, who were not
likely to vote and 874 who were likely to vote. I've read the poll. I've read
the research methodology. No other information appears on the polling agency's
page to explain who those 517 folks were (I mean demographically, rather than
personal identity) or WHY their responses are a part of the survey.
The questions not answered in the survey or
its methodology include whether, and to what extent, the inclusion of those NOT
LIKELY TO VOTE in the survey, skewed the outcome. But given the substantial
shift from ABC/WaPo's survey of two weeks ago, which was of LIKELY VOTERS ONLY,
it seems reasonable to ask about that group, and in the absence of a forthright
explanation, it seems reasonable to conclude that was the purpose of the group’s
inclusion in the poll.
