Noto Bene: The duty of candor requires that I mention that the October 23, 2016, poll included 874 LIKELY voters. But it also included 517 NOT LIKELY TO VOTE participants.
Monday, October 24, 2016
The ABC/Washington Post Poll of People Not Likely to Vote
Here's another example of using oversampling to steer the outcome of a poll.
ABC News Washington Post Tracking Poll today is being reported to show that Clinton has opened a 12 point lead over Trump. Fascinating stuff. Odd, too, considering that less than two weeks ago, a similar poll reported by ABC/WaPo showed Clinton leading by 6 points.
Let's call today's poll the "new poll" and the other poll the "old poll."
The old poll was conducted by interviewing fewer than 800 "LIKELY VOTERS."
The new poll was conducted by interviewing just over 1300 subjects, of whom 874 were LIKELY VOTERS.
The results of the new poll, however, are not the results of just those 874 likely voters. That poll interviewed 1391 people. Nearly 500 people were interviewed and their responses included in the results of the poll despite the fact that they were NOT LIKELY TO VOTE.
The total result of the poll included 874 folks that are likely to vote and FIVE HUNDRED SEVENTEEN respondents who ARE NOT LIKELY TO VOTE.
NOT LIKELY TO VOTE.
Think about the meaning of that phrase and how it connects to the matter with which we tend to ASSUME news organizational polling is concerned. Maybe that is the place to start.
Why do you look at news reports of polling?
Is it a search for confirmation? Sort of an internal dialog, "See, the majority of folks agree with me" kind of thing?
Is it an effort to see into the future? Sort of a "Well, let's see how this is going to turn out? kind of thing?
I think these are two of the most common responses to expect when you get honest responses from folks as to why they are interested in such polls.
But suppose what you are looking for in polling is not what campaigns and/or news agencies are seeking?
You realize that "push polling" exists. Those polls that, rather than gauging the temper of the people seek to put fire under them or dump cold water on them.
Let's not jump to any conclusions yet. But, suppose a polling agency and its partners wanted to steer the outcome of a poll. If they did, which approach do you think might be more effective. I'll give you two choices.
After confirming that the interview subject fits poll requirements (likely voter or not; party affiliation or not; racial and cultural demographic or not), ask one question:
"In a two way race between Hillary Clinton or Donald Trump, for whom would you vote?"
Again, after confirming that the interview subject fits poll requirements (likely voter or not; party affiliation or not; racial and cultural demographic or not), ask three questions:
"First, Donald Trump says the election may be rigged through election fraud. Do you think he is expressing a legitimate concern or that he is making excuses for a possible unfavorable outcome?"
"Second, Donald Trump refuses to say if he will accept the outcome of the election. Do you approve or disapprove of his remark?"
"Third, In a two way race between Hillary Clinton or Donald Trump, for whom would you vote?"
Now, you make your own judgment. But for me, asking a couple of good ground penetrating bomb blast questions might just be the kind of thing that deforms outcomes that might otherwise be gotten by simply asking for whom the likely voter is likely to vote.
But Choice A above is not what Langer Associates did in its polling for ABC News and the Washington Post. Instead, it took Choice B, peppering participants with those questions.
The ABC/WaPo poll also engaged in modeling. This approach looks at some past election, determines participation percentages by party and then skews its sampling according to those numbers.
Here, the Langer agency skewed its sample. Here's how they described their skew: "Partisan divisions are 36-27-31 percent, Democrats-Republicans-independents." In other words, the agency set itself a quota task: 36% of those interviewed must be Democrats, 27% must be Republicans, 31% must be Independents. These kinds of partisan divisions are included in most polls. The exact percentages change. For example, why 36-27-31 when the Pew Center says that, currently, the breakdown in partisanship is 33-29-34?
You can see, I'm sure, then, that the specific, formulaic breakdown used in selecting polling participants can skew the numbers, and it becomes important, if polls are something you have to deal with or choose to deal with, to KNOW what the pollster is using for partisan division, and to know WHY they are using any particular spread.
I'm going to wrap here by returning to that 517 number.
Remember, this poll was one conducted by interviewing 517 people, according to the polling methodology, who were not likely to vote and 874 who were likely to vote. I've read the poll. I've read the research methodology. No other information appears on the polling agency's page to explain who those 517 folks were (I mean demographically, rather than personal identity) or WHY their responses are a part of the survey.
The questions not answered in the survey or its methodology include whether, and to what extent, the inclusion of those NOT LIKELY TO VOTE in the survey, skewed the outcome. But given the substantial shift from ABC/WaPo's survey of two weeks ago, which was of LIKELY VOTERS ONLY, it seems reasonable to ask about that group, and in the absence of a forthright explanation, it seems reasonable to conclude that was the purpose of the group’s inclusion in the poll.