Previous Month | RSS/XML | Current
Just two months ago, there was an unprecedented case of a "pollster" who faked polls1. The fake pollster in question claimed that the hoax was a "social experiment" intended to show how polling misinformation can spread without independent verification―it certainly showed that it can spread a little, but its spread was soon halted and the polls exposed as hoaxes. This was a heartening result.
Now, another "pollster" has revealed that it also faked recent polling results, giving the same excuse for doing so as in the previous case. This one was called "Wedgewood Polls" (WP) and it seems to have had more success at fooling people than the previous one, Median Strategies (MS), which turned out to be just one 21-year-old recent college graduate with no obvious polling background. MS produced just three fake polls whereas WP seems to have put out seventeen2.
In the case of MS, I could find only one minor California publication that reported one of its fake polls. WP, in contrast, took in several major publications, including Newsweek, which had to retract three articles and correct additional ones3. This is discouraging news.
In addition to news sites, several polling aggregators―outfits that average multiple poll results―included WP's fake polls, thus skewing their averages. As far as I know, no aggregator included MS' results in its averages. I've recommended polling aggregators in the past4, because averaging can mitigate against the biases of individual pollsters and the occasional outlier result. I still do recommend them, but they obviously need to be more vigilant before adding the results of new pollsters to their averages and, after this debacle, I expect they will be.
I don't know how much WP's fake polls affected the aggregators' results, or the general perception of the candidates' popularity, but at least one candidate has claimed that one of WP's fake polls sabotaged her campaign. In the case of MS, I speculated that the motive behind the so-called social experiment might have been to create a bandwagon effect5. Given that two out of three of its fake results heavily favored Democrats―I haven't been able to find out about the third one―they look like an attempt at a bandwagon for the Democratic candidates. If it were actually a "social experiment", one would expect the results to split evenly between the two major parties; when all or most of the phony polls favor one party, and that heavily, that's a strong reason to suspect an intentional effort to affect public opinion.
Similarly, WP's faked results seem to have been calculated to promote Democratic candidates. Left-leaning news site Slate6 writes:
Over the past few months, an upstart political polling firm kept delivering good news for Democrats―and especially those on the left. Across more than a dozen surveys, it found progressives leading in key primary races and showed other Democrats in striking distance of winning traditionally Republican states. … Wedgewood claims its deception "was never intended to sway an election or permanently mislead the public," but it may also have wanted to make certain Democrats look more electable….7
In both these cases, the pseudo-pollsters were facing imminent exposure when they 'fessed up, which looks like they got caught trying to influence elections with bogus polling results and then gave the "experiment" excuse.
If this trend of hoaxed polls continues, just imagine what the polling leading up to the general election is going to be like two years from now. Polling is already in disrepute and the last thing its reputation needs is a spate of fake polls. If there's just one more polling hoax this year I'm going to decide, in accordance with Goldfinger's Principle8, that it's enemy action.
Update, 10/25/2026: That was fast! The polling outfit SoCal Strategies (SS) has disappeared in much the same way as Median Strategies and Wedgewood Polls before it9: its website now returns a 404 error page, and its social media presence seems also to have been removed. So, it must have been engaged in some kind of shenanigans―why else suddenly disappear? However, as I write, we don't know exactly what kind of mischief it was up to.
As in the case of the previous two polling "firms", SS was under recent scrutiny due to some dubious polls when it decided to do a bunk, which is as much as an admission that something was wrong with those polls. In contrast with MS and WP, both of which were new outfits, SS was a comparative veteran. Both MS and WP started up in the middle of this year, whereas SS had been polling since at least 2024, and may have been a reputable pollster at that time.
Strangely, SS shut down in May of this year, but then continued to release three supposed polls, so it was a zombie pollster. It was these polls that were suspicious enough to inspire the inquiry that led it to turn off the lights, lock the doors, and flee without leaving a forwarding address.10
We've seen that the faked polls put out by both MS and WP favored Democratic candidates, sometimes heavily; what about the three "polls" put out by SS since its closure in May? The first of these, dealing with the Michigan senate race, was released at the end of July and showed Democratic candidate Haley Stevens two percentage points ahead of Republican Mike Rogers11. Real Clear Polling's (RCP) average currently has Stevens ahead by only 1.7 percentage points12, so SS' result was right on the money.
The remaining two "polls" were released in mid-September and both concerned the state of Texas. In the state's senate race, SS showed the Democratic candidate, James Talarico, five percentage points ahead of the Republican, Ken Paxton. RCP's average for the same date had Talarico ahead by only 2.7 percentage points, so SS' result was on the high side but not unreasonably so13. Finally, in the Governor's race, SS had the Republican candidate ahead by eight points, while RCP's average leans in the same direction by only 3.4 points. Again, a little extreme but in the right direction14.
A person connected to WP was a former employee of SS, so at least those two "pollsters" were related to each other. It's hard to believe that it's just a coincidence that three different outfits have disappeared within a couple of months of each other. In accordance with Goldfinger's Principle, my working hypothesis is that these three "pollsters" were connected and probably working in concert to move the polling aggregators' averages, either to create a bandwagon effect or to make money in the prediction markets.
Just as the "social experiment" explanation doesn't explain why the thumbs of MS and WP were on the scale of the Democrats, neither does the hypothesis that they were trying to manipulate prediction markets, since they could do so just as well by putting their thumbs on the other scale. So, those results favor the bandwagon hypothesis. However, at this point, we don't know exactly what was wrong with SS' polls, and the results don't support either hypothesis, since they were neither skewed towards one party, nor skewed heavily enough to have influenced the markets.
This is now clearly a crisis for the polling industry, and all of the polls for the upcoming midterm election need to be taken with a shaker of salt. Given what's happened in the last couple of months, who knows whether other polls and pollsters will turn out to be fakes? The legitimate polling organizations―if there are any left―need to start an emergency investigation and clean house if they want to restore confidence in their industry.
Notes:
On a cool autumn day, Montgomery Banks2 was back at the carnival, and this time he had a spinner on top of his counter. The circular face of the spinner was divided into two sections, as shown: a large red section, making up three-fourths of the spinner's face, and a small green section accounting for the remaining one-fourth of the face. Attached to the center of the spinner was a rotating pointer.
"Step right up, sir or madam, as the case may be," he said to the crowd strolling past his booth, "and take it for a spin. Looky here, this spinner is completely fair and you can even spin it yourself. If you do and the arrow stops on the green section, you win a dollar; if it stops on the red section, you lose a dollar.
"Now, I know what you're thinking. There's only a one-quarter chance of you winning a spin, which means there's a three-quarter chance of losing, but I'm offering you an even-money bet. That ain't a fair bet, is it? No, sir.
"I tell you what I'm gonna do: If you don't win on the first spin, I'll let you try again, and if it stops in the green the second time, you win a buck. That's right, a second spin at no additional charge! That's two spins for a buck, and if you get in the green on either spin, you win. Two times one-quarter is one-half, so that's a fair bet, ain't it?"
Is the bet that Monty's offering fair at even money? Would you be a "sucker" to take the bet?
Extra Credit: If the bet is not fair, what is the real probability that the player will win?
What is the probability of the player losing both spins?
No―and this should be unnecessary to say―the bet isn't fair.
Explanation: Monty is correct that 2 × ¼ = ½, but that's not the right way to figure the probability of winning this bet. The easiest way to determine the probability of winning a bet that involves more than one trial is to figure the likelihood of losing each trial, then the probability of winning at least one trial is one minus that of losing each trial. Since the probability of losing a single spin is ¾, and the spins are independent of each other―that is, whether you win or lose a spin doesn't affect another spin―the probability of losing both spins is ¾ × ¾ = 9/16. Therefore, the probability of winning at least once is 1 − 9/16 = 7/16. A fair bet would be 8/16 = ½, so Monty's advantage in this bet is 1/16, or 6.25%.
Disclaimer & Disclosure: Like Monty, the above story is fictional. I came across a version of this gamble in Steven E. Landsburg's book Can You Outsmart an Economist? (2018), puzzle 2.
Notes:
Assuming that the following statements are either true of false, which if any are true?
Could both statements 2 and 4 be true?
Only statement 5 is true.
Explanation: Statement 1 cannot be true since then statement 2 would have to be true, which would make statement 3 both true and untrue. So, statement 3 is also untrue, since 1 is not true. This leaves statement 2: is it true or not? If it were true then statement 4 would be true, since statements 1 and 3 are untrue. However, 2 claims that 4 is not true, which is a contradiction. Thus, 2 is untrue, which means that 4 is also untrue. Therefore, statement 5 is true, since none of statements 1-4 is true, which means that statement 6 is untrue.
Disclosure: This puzzle is based on the second one from the following article: Alex Bellos, "Can you solve it? I've got a little (logical) list", The Grauniad, 10/8/2018. I simply inverted the puzzle in the article, reworded it to make it clearer―I think―and swapped the "above"s and "below"s. To my surprise, it worked just as well upside-down!
I was puzzled by the following headline:
What puzzled me about this headline is how anyone who was actually sucked out of the window of an airplane could claim anything. This could only happen at high altitude because the cabin is pressurized when a plane is at such an altitude that the atmosphere is becoming too thin2. If cabin pressurization is suddenly lost at such an altitude, which would happen if a window failed, the difference in pressure between the cabin and the atmosphere outside the plane could suck someone through the window3. At such an altitude, if a passenger were pulled out of the window, he would surely fall to his death unless he was wearing a parachute. Assuming that the passenger wasn't actually sucked out of the window, what did he claim?
Despite the headline, and as we learn in the article beneath, the passenger claims only that his head and shoulders were sucked out of the window, while his wife held on to his legs to keep him from going completely out. The airline, Ryanair, is counter-claiming that no part of him was ever outside the window; why that matters is not clear.
No doubt brevity is important in headlines but not at the expense of accuracy. There was no necessity for the editor who wrote the above headline to give the impression that a man survived being sucked through an airplane window, as other editors were able to write headlines for their coverage of the same story that included the adverbs "almost", "nearly", or "partially" to modify the verb phrase "sucked out"4. In particular, The Grauniad itself, from which the above headline was taken, managed to get it right a couple of months earlier:
Despite its unlikeliness, I wondered whether anyone ever survived being sucked out of an airplane window. Here's another headline that sucks:
No, Snopes, he really didn't. If you read the article beneath the headline―which I recommend as it's quite fascinating―you find out that the pilot wasn't sucked out of the plane. Like the man in the more recent story, he was partially sucked through the window while other crewmembers held onto his legs until the plane was able to land. If he had gone all the way through the window, he would surely have been killed.
It's rather disappointing that Snopes, of all sites, ran such a tabloid-style headline for its story. I was excited to read how someone was sucked out of a plane by explosive decompression yet somehow survived the fall, only to discover it was much the same story as the Ryanair one.
I've found no evidence that anyone sucked completely out of a pressurized airplane cabin has ever survived. Those who lived through a violent decompression were saved by neighbors who held onto them. Thankfully, such cabin decompressions are so rare that you're more likely to die from drinking the piping hot airline coffee―see the previous entry. Whether the tiny risk of being sucked out of the plane is worth choosing an aisle seat and wearing your seatbelt while in flight, I'll let you decide.
Notes:
Most of the previous entries in this series on the supposed health effects of coffee have been about claims that coffee is good for you, but rarely a study claims that drinking coffee is the opposite1. Here's a recent one.
A study claiming that drinking very hot coffee increases the likelihood of cancer of the esophagus has recently been reported by several news outlets2. However, according to the study in question, the problem is not with coffee itself, but with drinking any very hot liquid, so that hot tea and hot chocolate are included.
The headline claim is that drinking "scalding" hot beverages can triple the risk of esophageal cancer. Tripling the risk certainly sounds bad, but thrice what? It's a meaningless claim if we don't know what the risk of getting the cancer is for those who don't drink very hot beverages: three times zero is still zero.
The tripling claim is a relative risk or risk ratio (RR), that is, the ratio between the people in the study who drank "very hot" drinks and ended up with cancer of the esophagus to those who didn't drank such hot drinks and still got it. The RR tells us that there were three times as many in the former group as in the latter, but how many were in the latter? Before we can evaluate the RR claim, we need to know the absolute risk (AR) of esophageal cancer, that is, the risk of getting it without drinking superheated beverages3.
Cancer of the esophagus appears to be rare, but how rare? I've known more than one person who had cancer, including a couple who died from it, but I've never met anyone with cancer of the esophagus. The only public figure I've ever heard of who had it was Christopher Hitchens, who died all too young from complications of the cancer4. Did Hitchens drink extremely hot coffee or other beverages? I don't know, but he was notorious for smoking cigarettes and drinking copious quantities of alcoholic beverages, presumably cold5. As many of the news articles point out, cigarette smoking and drinking alcohol are higher risks for esophageal cancer than drinking very hot beverages.
Now, I'm not saying there's no risk of getting cancer of the esophagus―obviously, there must be some risk―but if the risk is very low then tripling it will produce a risk that is still low, maybe very low. This is important to know because life is inherently risky, and we need to know just how risky an activity is to rationally decide whether to do it. For instance, skydiving is risky, but apparently it's enough of a thrill for some people that it's worth the risk. Similarly, there may be some people who enjoy drinking scalding hot coffee so much that a tripled risk of cancer is worth it for them; Hitchens, for instance, claimed that smoking and drinking were worth not just risking it, but actually getting cancer.
In these entries on how health claims are reported, I'm usually quite critical of the reporting, but most of the reports on this study were better than usual. Of course, that's a low standard since so many news reports of new studies simply rewrite the press release without asking questions. However, some of these reporters interviewed an expert who was not connected to the study itself, who then pointed out that the absolute risk is less than 1%, and that smoking cigarettes and drinking alcohol are greater risks than drinking very hot drinks. So, the reporters, to their credit, did some actual reporting.
That said, "less than 1%" still sounded rather high to me, since that would mean one out of a hundred people gets esophageal cancer. Actually, "less than 1%" refers to the percentage of cases of cancer that are of the esophagus, but not everybody gets cancer. What we really need to know is the prevalence of esophageal cancer in the population. It turns out that the rates are quite different for men and women, though both rates are below 1%. For men, it's about .76% and, for women, only around .23%6. Putting these two figures together, the prevalence is approximately .5%7, which is indeed less than 1%. Tripling this rate by drinking searing-hot coffee leads to a risk of 1.5% for people in general.
Whether this is enough of a risk to justify letting your coffee cool off a bit longer before drinking it, I'll leave to your judgment.
Notes:
In a foreword to a book about the use of mathematics in magic tricks, Martin Gardner wrote: "You will learn how parity (odd or even) can play a roll in magic as well as provide powerful shortcut proofs.1" How do you a play a roll? If you're a drummer, you can play a drumroll, but what does that have to do with magic tricks? You could play a roll on Halloween, just as some people don costumes as hot dogs or bananas2, in which case you'd be playing the role of a roll. Which brings us, of course, to what Gardner meant to write: parity plays a role in some magic tricks and mathematical proofs. Whether Gardner himself misspelled "role" or the misspelling was introduced later, it was ultimately the responsibility of the book's editor to correct it.
"Role" and "roll" are homophones, that is, words that sound the same but are spelled differently and have different meanings. "Role" is a noun for the act of playing a part in something, such as a play or other performance3: the role of Hamlet, for instance. By extension, it can mean the playing of a part in other types of event or situation: for example, the role of magician in a card trick.
"Roll" is a highly ambiguous word with a long list of distinct but related meanings4. It's even ambiguous in grammatical category, since it can play the role of both a noun and a verb, but its use as a verb seems to be primary. "To roll" is to turn something over, or move something by turning it over. As a noun, a "roll" is an object that either works by turning over―such as a roll of toilet paper―or is made by turning over something―such as a cinnamon roll, which is made by rolling dough into a cylinder and then slicing off an individual roll.
One reason why these two English words are spelled so similarly and, thus, so easily confused, is that they descend from a common ancestor, namely, the Old French word "rolle"5. This word seems to have come into English first as "roll", referring to a roll of parchment, which was the predecessor of the modern paper book. Later, "role", as in "playing a role", came from the use of rolls of paper as promptbooks for the actors' lines in a play.
As with many of the confusable word pairs in this series, the confusion goes primarily one way, namely, "role" is misspelled "roll", which may simply be because the latter is the more familiar word. Misspelling "role" in this way is a common enough mistake that I've seen it before, yet not a single one of the reference books I usually consult mentions it.
If you don't want to play the "roll" of an unlettered person, spell it "role".
Notes:
Something calling itself "Median Strategies" (MS) put out press releases claiming to have done three different polls: one for the Democratic primary for governor of Wisconsin, another for the contest for the governor of Nevada, and finally one for the race for mayor of Los Angeles1. The last one received the most media attention by claiming that Democrat Karen Bass, incumbent mayor of Los Angeles and candidate for re-election, had a twelve percentage point lead over challenger Nithya Raman. Bass' campaign trumpeted the results before it was revealed that all three polls were hoaxes2.
Political partisans frequently claim that polls whose results they dislike are fake, or at least rigged; for instance, President Trump loves the word "fake", calling news he doesn't like "fake news" and poll results he hates "fake polls"3. While there have been "push polls" in the past, which could be rightly called "fake"4, outright hoaxes have been rare.
Since this news may well lead to a decline of trust in polling, I wondered whether any polling had been done to measure the level of trust. Surprisingly, there was such a poll just last week released by a real pollster as opposed to a fraudster: half of respondents said they don't trust polls, with only 28% saying that they did5. According to the pollster, this result has not significantly changed in the last decade, so the pollster has been polling on this topic for at least that long. Assuming that it keeps asking the same question―and I hope it does―it'll be interesting to see whether the current news leads to a loss of trust.
What was the motive for the hoax? MS claimed in its announcement that it was "a short-term social experiment examining how purported polling information could enter and spread through the political information ecosystem without independent verification"6. Every report on this hoax that I've seen seems to take this claim at face value, but that's the same mistake that those who reported on the poll as if it were genuine made. Given that MS has lied about everything else, why should we believe this claim?
Assuming that the claim is true, this "social experiment" had mixed results. The alleged poll of Los Angeles voters received the most media attention probably because the Bass campaign publicized it. So, there was at least some "spread through the political information ecosystem without independent verification" of that one. This led some news media to try to investigate MS, and that seems to have triggered its public confession and the whole hoax only lasted a few weeks. I'm about as cynical about our "political information ecosystem" as anyone, but this is a reassuring result. It's not as easy as you might have thought to get away with hoaxing poll results.
If this was a "social experiment" by a scholar or scholars, one would expect that those responsible would immediately reveal their identities, and a publication of some sort should ensue. There is a recent history of scholarly hoaxes, starting with the Sokal hoax and followed by the so-called "Sokal squared" hoaxes7, but in both cases those involved revealed their identities and explained the points of the hoaxes.
If it wasn't a "social experiment", what was it? Perhaps it was an attempt to influence the elections by injecting hoaxed polling data that might contribute to a "bandwagon effect"8. This might explain why the fake polling numbers for Bass were so lopsidedly in her favor, while contemporaneous polls had the race between her and her opponent closer. As a result, the "poll" was an outlier, thus drawing attention to the formerly unknown "pollster".
I haven't been able to find any information on the hoaxed Nevada poll, but the poll on the Democratic primary for governor in Wisconsin was similarly skewed, showing Francesca Hong with a 23-point lead in a race that she subsequently lost9.
In its last statement, MS claimed: "No individual involved with Median Strategies received compensation or financial benefit from the project, and the project was not funded, commissioned, or directed by any candidate, campaign, political party, PAC, media organization, or other outside entity." Again, why should we believe this disclaimer? Even if true, it doesn't rule out that an individual or individuals involved in it―and, thus, inside entities―wanted to help one or more of the candidates and tried to do so by means of the hoax.
Unfortunately, we'll probably never know the correct explanation of this hoax unless the party or parties behind it are identified.
Update, 8/21/2026: That was fast! When I wrote the above, whoever was behind "Median Strategies" was unknown, but now the perpetrator, a young man, has confessed to making up the polling results with the help of "artificial intelligence". As in the final statement put out by MS, he claims to have simply wanted to see whether hoaxed polling results would be picked up by the news media. As I pointed out above, the stunt was only partially successful. He also claims, as he did in that statement, that he didn't make any money from it, but why should we believe him? The Grauniad tracked him down based on his domain name registration for MS' website, so he didn't do a very good job covering his tracks10.
Notes: