Nate Cohn, who covers politics and polling for the New York Times, wrote an article last month (6/3/22) comparing the results of national polls with actual ballot measures in states. His major theme was that national polls may be unreliable when they show high levels of support for various policies. The reason: When those same policies are voted on at the state level, they uniformly perform worse than what the national polls show.
He suggested to supporters of gun control in particular: “Their problem could also be the voters, not just politicians or special interests.”
While it’s always prudent to be skeptical of polls in general, I believe Cohn goes too far in dismissing the utility of national polls, and implying that the votes on state initiatives and referendums are a better indicator of what the public wants.
Cohn’s initial focus was on guns. He noted that widespread public support for background checks, as revealed by national polls, was not reflected in state balloting on the same issue:
When voters in four Democratic-leaning states got the opportunity to enact expanded gun or ammunition background checks into law, the overwhelming support suggested by national surveys was nowhere to be found. Instead, the initiative and referendum results in Maine, Washington, Nevada and California were nearly identical to those of the 2016 presidential election, all the way down to the result of individual counties.
A bit later in the article, he asserted:
Hillary Clinton fared better at the ballot box than expanded background checks in the same states, most on the same day among the same voters.
As it turns out, Cohn appears to be completely wrong in the latter statement, and partially wrong in the previous one.
Gun questions vs. Hillary Clinton
Here are the results of the 2016 ballot questions in Washington, California, Nevada and Maine (all dealing with gun or—in the case of California—ammunition background checks), compared to the 2016 presidential election results in Washington, California, Nevada and Maine:
In no state did the gun measures fare worse than Hillary Clinton. It is true that in three of the states—California, Nevada and Maine—Clinton’s vote total was similar to (though slightly lower than) the level of gun measure support. In Washington, the gun measure outperformed Clinton by 16.9 percentage points.
Cohn compounded his apparently faulty observations by concluding:
The possibility that one of the most popular policies in polling could run behind Mrs. Clinton at the ballot box raises important questions about the utility of issue polling, which asks voters whether they support or oppose certain policies. While these questions probably tell us something about public opinion, it may tell us quite a bit less about the political landscape than many assume.
Well, as we saw, the gun measures did not run behind Clinton at the ballot box. But do the polls tell us “quite a bit less about the political landscape than many assume”?
National polls vs. state ballots
For evidence on this point, Cohn referred to a model developed by two academic researchers, professors Christopher Warsaw of Georgetown University and Devin Caughey of MIT, which showed that the gun-related ballot measures in each state all received lower support than what national polls would predict.
In California (in 2016), the model predicted a 91% majority, while just 63% voted for the measure. The comparable figures for the other states: Washington (2014*: 81% projected, 59% received), Nevada (2016: 86%, 50%), Maine (2016: 83%, 48%).
Those projected levels of “expected support” for the state ballots, ranging from 81% to 91%, seem implausibly high—not necessarily because of a faulty model, but because the national polls that provided the input to the model were probably implausible themselves.
That’s because, as I’ve noted previously (FAIR.org, 2/11/22, 5/10/22), pollsters, by the use of forced choice questions and by failing to measure intensity, can exaggerate how many people are engaged enough to have a meaningful opinion. Answers from respondents who don’t really care much about an issue are combined with answers from more committed respondents, giving the illusion of more engagement—and often more support for a policy—than actually exists. Most polls fit that description, and if used in the model could lead to inflated results for expected support.
Cohn acknowledged this possibility:
Those polls have shortcomings of their own. Many voters hold relatively weak views about specific policy items. They may be especially likely to say they “support” policies in a survey, where “acquiescence bias” can lead respondents to agree with what’s being asked of them. Those attitudes might shift quickly once an issue receives sustained political attention.
He also acknowledged that there are many other reasons why national polls may show a much more favorable public opinion than what state ballot votes produce:
It would be wrong to say that the results [discrepancies between ballot measures and national polls] simply prove the polls “wrong,” strictly speaking. Initiative and referendum results are not a perfect or simple measure of public opinion. The text of the initiatives is different and more complex than a simple national poll question. Some voters who may support a proposal in the abstract may ultimately come to oppose its detail. The context is very different as well. The vote follows a referendum campaign that can shift public opinion.
And the act of voting to enact an initiative into law carries far more responsibility and consequence than a carefree response on a survey. When in doubt, many voters may adopt a lowercase “c” conservative position in the ballot box.
All together, it is no surprise that initiatives and referendums tend to underperform their support in the polls [emphasis added].
And yet—and here is where his argument went off the rails—he still insisted that there may be something “wrong” about the national polls. When tested by ballot measures, “the overwhelming support suggested by national surveys was nowhere to be found.”
Initially, he argued that “guns are just different.” The difference between national polls and gun control referendum results, he wrote, “is an order of magnitude larger than on other issues.”
But later he wrote that “voter identification requirements and parental notification for abortion also receive overwhelming support in the polls,” and “these initiatives have underperformed at the ballot box by nearly as much as background checks.” The order of magnitude apparently disappeared.
Looking more broadly, Cohn viewed the discrepancies between ballot measures and national polls as a general phenomenon: that whenever a large majority appears to exist, it is prone to evaporate during a campaign or during subsequent public debate. One example: “Healthcare reform started out as popular, until the Affordable Care Act was actually proposed and debated.”
Here Cohn has abandoned altogether his focus on ballot measures vs. national polls, and on guns as an outlier. Instead, he sees a common pattern where opinion—solely as measured by national polls—shifts significantly as public debate occurs.
Despite all his compelling arguments why one should not expect ballot measures to reflect national polls, Cohn concluded his article by claiming that “a broad challenge to issue voting” remains, and that it is
more inconvenient for progressives than conservatives…. If the public’s operational liberalism functions only in an interview with a pollster, not at the ballot box, it may not count for much.
National polls that differentiate opinion from non-opinion, and anchored opinion from top-of-mind opinion, are important sources of what the public thinks. They “count” by providing a guide to elected representatives about what the public wants, and more broadly by demonstrating how well and when the government responds to the will of the public.
Cohn has not shown that such polls “don’t count for much.” What he has shown is that ballot votes and national polls typically measure opinion in different ways, among different groups of citizens, at different times in the development of public debate. And, as he acknowledged at one point, “it is no surprise” that they diverge.
Both sets of results tell us much about the public and about the state of American democracy.
*In his initial reference to Washington state, Cohn mentioned the 2016 ballot measure dealing with guns, because he could compare Clinton’s support with the ballot results. But in his second mention of a Washington ballot measure on guns, he referenced the 2014 results. Why he switched to 2014 is not clear.
ACTION ALERT: You can report errors in New York Times news coverage at email@example.com (Twitter: @UpshotNYT). Please remember that respectful communication is the most effective. Feel free to leave a copy of your communication in the comments thread.
Featured image: Gun show, Shelby, North Carolina (cc photo: Brittany Randolph).
The post National Polls, State Ballots Measure Different ‘Public Opinions’ appeared first on FAIR.
This content originally appeared on FAIR and was authored by David W. Moore.