You are on page 1of 2

TO:

FR:
DT:
RE:

Interested Parties
Jarrett Lewis, Nathan Klein
July 20, 2015
GOP Debate Criteria Polling Flaws

Recently, the format in which participants for the first two Republican debates are selected has
come under scrutiny. Fox News and CNN are undoubtedly in a tricky position with regards to how
they select the participants including 17 candidates on a debate stage is simply not possible.
However, the process by which they are selecting the 10 participants has significant flaws,
particularly for the lower tiered candidates.
The main element for entry into the debate (excluding proper filing of FEC paperwork and
submission of federal and state fees, etc.) is to be among the top-10 candidates in a ballot test of
Republican voters in an average of the most recent 5 national surveys through August 4.
In light of todays release of a national poll of only 210 potential Republican primary voters a poll
which could very well have a role in deciding who qualifies for that stage we felt that it is critically
important to have an honest conversation about the three fundamental problems with this
approach:
1) These national polls have tiny samples of primary voters. The sample sizes of the
surveys that currently make up the Real Clear Politics average of Republican primary polls
average 341 voters. The five surveys that currently make up the Real Clear Politics average
(with Republican primary voter sample size and Margins of Error shown) are:

Washington Post, n=210, MoE 6.8%


Fox News: n=389, MoE 4.9%
USA Today/Suffolk: n=349, MoE 5.2%
Monmouth: n=351, MoE 5.2%
CNN: n=407, MoE 4.9%

A survey of only 341 voters is less than the average political consultant would use for a
contested Congressional race, let alone an extremely complicated and dynamic Presidential
nominating contest. Quinnipiac, for example, interviewed nearly double that amount (666)
for a statewide poll of caucus-goers in Iowa.
2) Geographic representation means that many of these voters are not in states critical
to the primary process. The same surveys include voters from states that have not voted
in a meaningful Republican presidential primary contest since the 1970s, while significantly
underrepresenting the early states of the primary.
The CNN national survey featured 407 Republican or Republican leaning voters. Assuming
the number of respondents from each state represented the states portion of the U.S.
population (i.e., New York has 6% of the U.S. population, so 6% of the survey would be
made up of respondents in New York), there would be only 16 respondents in the CNN
survey from the first four states of Iowa, New Hampshire, South Carolina and Nevada.
For the 10 states planning to hold their nominating contest in May or June, more than five
times as many voters, 89 in total, would be included. California alone, bereft of a meaningful
GOP Presidential primary since 1976, would contribute 49 of the voters.

Page 1 of 2

The polls that make up the debates should not be solely based on the first 4 states of Iowa,
New Hampshire, South Carolina, and Nevada, but few can argue that the race will look
significantly different after these states hold their contests. Yet, as of today, they comprise a
tiny fraction of the surveys being used to make these decisions, while states that very likely
will not matter comprise a sizeable chunk of the sample.
3) Definition of a primary voter is inconsistent, and often incomplete. In several of the
surveys, respondents were not asked whether they planned to vote in the upcoming
primary. They were simply directed to the ballot test if they identified as a Republican (or
Democrat, in which case they were steered towards the Democratic primary ballot).
Voters who participate in primaries are different than those who participate in a general
election. In Iowa in 2012, roughly 1 in 5 Iowa Republicans participated in the closely
contested Iowa Caucuses.
There is no perfect way to identify a primary voter for these purposes. Commonly used
techniques include examining past participation, asking knowledge screens, and relying on
self-reporting. Regardless of the technique, efforts should be made to find actual voters, not
just registered voters of the same party affiliation.
Bottom Line: There is no perfect solution and the networks have a daunting task, but entry into the
debates could provide an enormous opportunity for one of the lesser-known candidates. Deciding
who gets a ticket in based on national surveys with 300 or 400 voters is simply insufficient.
At the very least, these surveys should be considerably larger and more targeted. And, dont forget
that sometimes voters want to hear from the candidates they are not as familiar with, to learn
enough to make an informed decision. Consider asking not only for whom they would vote, but
whom they most want to hear from in the debates, use a two-tiered first choice/second choice
question, and ask the voters who will be actually voting in these contests.
Yes, doing these things costs more money. Doing anything the right way costs more money. But, we
believe it is worth it after all, it is only the future of our nation at stake.

Mr. Lewis (@jarrettlewis) serves as Executive Director of Consumer Research at The Health
Management Academy. He previously worked as a polling analyst on Governor Mitt Romneys
Presidential campaign during the 2012 election cycle and at the Republican polling firm Public
Opinion Strategies in the 2008 cycle.
Mr. Klein (@nathankleindc) is founder and managing partner at Olive Tree Strategies, a political
polling and consulting firm. He was previously the Director of Polling and Analytics for the NRSC
during the 2014 election cycle and the polling manager for the Romney Presidential campaign during
the 2012 election cycle.

Page 2 of 2

You might also like