Previous Month | RSS/XML | Current
WEBLOG
March 6th, 2025 (Permalink)
Oh, Snap!
Here's a recent headline from Newsweek based on two "snap" polls1 conducted right after Trump's big speech a couple of nights ago:
Donald Trump's Congress Speech Was a Huge Hit With Americans2
These polls, which are the only ones I could find that were done since the speech, showed similar results: in one sponsored by CBS News3, 76% of respondents approved of the speech; in one from CNN4, 69% had a positive reaction.
The target population for both surveys was those American adults who watched the address, and not American adults in general. In public opinion surveys, a sample is taken of the "population", which is the group you want to know about, then the results for the sample are extrapolated to the population as a whole. In both of the snap polls, the population sampled was American adults who watched the speech, not American adults in general. For this reason, the poll results can only be extrapolated to the population of those who watched the address, and not to Americans as a whole, contra Newsweek. This is a fundamental point about sampling: a sample can only tell you about the population sampled, and not some other population, not even a larger one of which the sampled population is a subgroup.
Given that Republicans and Trump supporters are more likely to watch a speech by the president than Democrats and Trump opponents, adult address-watchers is a self-selected subgroup of the more general class of adults. This fact is reflected in the CBS survey in which slightly over half of those polled were Republicans and only a fifth Democrats; for the CNN poll, 44% were Republicans and also about a fifth Democrats. So, Republicans were over-represented in the samples and Democrats under-represented.
CNN's report did a good job of explaining this point:
Good marks from speech-watchers are typical for presidential addresses to Congress, which tend to attract generally friendly audiences that disproportionately hail from presidents' own parties. In CNN's speech reaction polls, which have been conducted most years dating back to the Clinton era, audience reactions have always been positive. The pool of people who watched Trump speak on Tuesday was about 14 percentage points more Republican than the general public.5
Another drawback of CNN's poll is its small sample size―only 431―and correspondingly large margin of error (MoE)―5.3 percentage points. In contrast, the CBS poll had a more usual sample size of 1,207 and MoE of 3.4 points. This is something to keep in mind about any snap poll, since such polls must be put together quickly and are, therefore, likely to have smaller samples and larger MoEs than standard polls.
Given the above considerations, there's not much news in either of the two snap polls. Mostly, what we learned is that people who watch Trump's speeches tend to agree with what he says. Big deal.
Notes:
- The word "snap" applied to a poll does not appear to have a technical meaning, but seems to refer to a topical poll taken quickly.
- Ewan Palmer, "Donald Trump's Congress Speech Was a Huge Hit With Americans", Newsweek, 3/5/2025.
- Anthony Salvanto, Jennifer De Pinto, Fred Backus & Kabir Khanna, "Poll on Trump's 2025 joint address to Congress finds large majority of viewers approve", CBS News, 3/5/2025.
- Ariel Edwards-Levy, "CNN poll: Trump address to Congress gets modestly positive marks, changes few minds", CNN, 3/5/2025.
- See the previous note. Paragraphing suppressed.