Parsing the polls: How to judge the validity, credibility of political surveys

It's important to understand how a poll is conducted, who's conducting it, how many people were surveyed and more.
AP Poll Election 2024
Posted

Regardless of all the opinions out there on political polling, between now and election day voters are about to hear plenty of talk about polls. But experts say people should try to examine those surveys the same way a professional pollster or researcher does.

To start, it's important to understand how a poll is typically conducted. Media companies, nonprofit organizations, academic institutes and political campaigns will often pay a professional firm to conduct a poll for them.

RELATED STORY | First national poll since Biden dropped bid shows tight race between Harris and Trump

Those polls can be conducted over the phone but they are increasingly using the internet to reach people. Professor Andy Crosby studies how polls are conducted at the University of California Riverside, and he says it's key for people to familiarize themselves with more than just a poll's headline.

"Take for instance this poll, the headline shows Vice President Harris leading former President Trump on the issue of the economy," Crosby said. "But note the margin of error. It's often listed much smaller than this onscreen. The margin of error of plus-or-minus 3.1% means it's mathematically possible that Trump is actually leading. A margin of error exists in every poll because pollsters aren't surveying every American."

Another big number to look for is sample size, or how many people were actually surveyed. It may not always be listed on the fancy graphics, but instead can be found in the poll's full report listed online. Crosby says a sample size of 1,000 or more is good, but anything less than that should be viewed with a bit more skepticism.

RELATED STORY | Americans agree democracy is under threat, but disagree on why

Meanwhile, it's also important to be on the lookout for companies who may have worked on the poll. In the case of the Harris-Trump poll previously mentioned, the logos of two polling firms are listed in the report: one is left-leaning and the other company leans Republican.

"If you see a poll that's only funded by a Democratic pollster or only by a Republican pollster, you may say, you know, maybe this poll is accurate but we know that it's funded by one of the political parties," Crosby said. "Maybe we want to use some caution on that."

If you really want to become an expert on polling, one step you can take is to familiarize yourself with who actually participated in the poll and look at how people were contacted — random is always best — but it may take some digging online to find this information.

Regarding the most recent poll from The New York Times and Siena College, you have to scroll to the very bottom of the report to find that nearly 2,000 people were randomly sampled, mostly by cellphone.