Errors of Unusual Magnitude
As I mentioned earlier this year1, the American Association for Public Opinion Research (AAPOR) created a "task force" to study the performance of last year's general election polls2. Its report is now out3 but I haven't had a chance to read it yet.
We already know that last year's polls were bad4, but just how bad were they? One thing that the new report did is to quantify how poorly the polls performed. The Washington Post's Dan Balz reports: "Public opinion polls in the 2020 presidential election suffered from errors of 'unusual magnitude,' the highest in 40 years for surveys estimating the national popular vote and in at least 20 years for state-level polls, according to a study conducted by [AAPOR].5"
I've mentioned in the past that there are a large number of polls taken in a presidential election year, but I've never known how many. Now, the AAPOR report tells us that the group examined 529 presidential polls from last year. I bring up the large number of polls not just because I think it's an absurd waste of time, effort, and money―though I do―but because given the usual confidence level of such polls, we can expect that 5% of them will be wrong by greater than the usual margin of error6. So, we could expect around 26 of last year's polls to be off by more than three percentage points.
Also, it was clear that the polls over-estimated support for Democratic candidates, but by how much? Based on the Real Clear Politics average of national polls, I calculated last year that they showed Biden winning by a margin of 4.3 percentage points of the popular vote greater than he won. According to the AAPOR report, this margin was only 3.9 points, so my calculation was not too far off.
According to The Post's report, the AAPOR report seems to have ruled out most explanations of the large error except:
One possible explanation is that Republicans who responded to surveys voted differently than Republicans who choose not to respond to pollsters. The task force said this was a reasonable assumption, given declining trust in institutions generally and Trump's repeated characterizations of most polls by mainstream news organizations as fake or faulty. "That the polls overstated Biden's support more in Whiter, more rural, and less densely populated states is suggestive (but not conclusive) that the polling error resulted from too few Trump supporters responding to polls," the report states. "A larger polling error was found in states with more Trump supporters."
The report makes an excellent point which has been an ongoing theme of these "Poll Watch" entries:
The report emphasizes that though often quite accurate, polls are not as precise as sometimes assumed and therefore given to misinterpretation, especially in the most competitive races. "Most pre-election polls lack the precision necessary to predict the outcome of semi-close contests," the report states. "Despite the desire to use polls to determine results in a close race, the precision of polls is often far less than the precision that is assumed by poll consumers."
When I've had a chance to read the whole report I'll update this entry or write a new one if I discover anything else in it worth writing about.
- What biased last year's polls?, 4/27/2021.
- "AAPOR Convenes Task Force to Formally Examine Polling Performance During 2020 Presidential Election", American Association for Public Opinion Research, 2/13/2020.
- Josh Clinton, et al., "Task Force on 2020 Pre-Election Polling: An Evaluation of the 2020 General Election Polls", American Association for Public Opinion Research, 7/19/2021.
- Post Mortem, 11/11/2020.
- Dan Balz, "2020 presidential polls suffered worst performance in decades, report says", The Washington Post, 7/18/2021. Subsequent quotes from this article; paragraphing suppressed.
- For the confidence level and margin of error of polls, see my: How to Read a Poll.
There ain't no such thing as free knowledge.
Quote: "Free knowledge from an encyclopedia―that would be a glorious thing. It is a shame that it is impossible. Knowledge is something that exists in minds, not texts. Reading a text will give you some ground for belief; it will not, by itself, actually give you knowledge. Still, we can speak loosely and say that encyclopedias contain what purports to be knowledge, and that is enough for me to love encyclopedias."1
Title: Essays on Free Knowledge
Comment: As the title of this newish book indicates, this is a collection of essays. I've already read some of them but am interested in reading the remainder.
Subtitle: The Origins of Wikipedia and the New Politics of Knowledge
Comment: I haven't written much about Wikipedia in recent years, but I've been a frequent critic of it on this weblog for over a decade. When it first began, I had some hope that it might turn out well, but was always skeptical of the approach taken to constructing it. Unfortunately, my skepticism appears to have been borne out by its subsequent development. I'll get into what's wrong with the approach later.
Author: Larry Sanger
Comment: Sanger is a philosopher and co-founder of Wikipedia, so he knows where the bodies are buried.
Summary: According to the book's table of contents, like all of Gaul, it is divided into three parts:
- The history and theory of Wikipedia
These are the questions that Sanger addresses in this part:
What makes an open, online collaboration succeed? … Should media, textbooks, and above all reference works aim to be neutral―or should they instead aim at what their editors claim is the objective truth? How should we organize people who are difficult to reconcile, who have different interests and agendas? How do we resolve disputes among anonymous people in open communities?2
Comment: I don't know the answers to any of these questions except the second: assuming that "neutrality" is not just another name for "objectivity", I think that reference works at least should aim for the truth. As far as I'm concerned, all truth is objective; the phrase "subjective truth" is just a fancy way of referring to an opinion or mere belief.
The first essay in this section and, thus, in the book, is one that I've already read: "The Early History of Nupedia and Wikipedia: A Memoir". This essay was written and published in 2005, when Wikipedia was still young, and its tone is more positive than Sanger's more recent writings. Perhaps this is because he was still too close to it to view it objectively, or time has not been kind to it. In a footnote to this essay added to the book, he writes: "By 2019, I had come to the view that Wikipedia is simply 'broken'"3. That doesn't say whether it took until two years ago for Sanger to realize that it was "broken", or that it wasn't broken until then.
I don't think that its early history is important to understanding what's "broken" about Wikipedia, nor does it show us how to fix it. It shows how we got here, but it doesn't show how to get out of here. For that reason, unless you're specifically interested in its history, I would suggest skipping over this essay, which is not to say that this detailed account isn't interesting.
The last essay in this section, "Why Wikipedia Must Jettison Its Anti-Elitism" is another that I've already read. It deals with the issue that I'm most interested, namely, expertise. Many of Wikipedia's supporters seem to believe that you don't need experts to produce an encyclopedia: that if you get enough ignoramuses together, they will produce knowledge. This sounds rather like the old probability theory chestnut that if you get enough monkeys typing away, eventually they'll produce the 11th edition of the Encyclopaedia Britannica. That's a sarcastic way of putting it, but the question is serious: how do you get knowledge out of ignorance?
- The politics of internet knowledge
In an age of instant answers from collectively-built databases, should we care about accumulating individual knowledge, or are mere information and collective knowledge good enough? What sort of special role, if any, do experts deserve in declaring "what we all know"? Is individual knowledge, built from books and individual study, somehow outmoded?4
Comment: To address the last question first: I don't see how "collective knowledge" can be constructed without individual knowledge from which to construct it. By an "expert", all I mean is a knowledgeable person, and not necessarily someone with a particular degree. If you know every Pokémon character and its abilities, then you're a Pokémon expert.
The answer to the last question answers the first. As for the second, I'm not sure what it's asking.
- Freer knowledge
In the final part I include three recent essays bemoaning the fact that free knowledge is in dire straits, now that, like social media, Wikipedia has abandoned neutrality and is used as a tool for social manipulation. … I conclude, in a brand new essay, that free information and knowledge on the Internet is under attack, and I ask how we can save it.5
Comment: Apparently, this section is at least partly promoting Sanger's new project: the Encyclosphere. I gather that the goal is to create an alternative internet encyclopedia that lacks the faults of Wikipedia. I wish the project well and would love to have a superior alternative to Wikipedia, which I would gladly use6, but I doubt it will work out. The problem is that Wikipedia has already grown so big that it's sucked all of the air out of the online encyclopedia reading room. Like such platforms as YouTube, Twitter, and Facebook, there's really only room for one such entity on the internet, and for better or worse we're probably stuck with it. I hope I'm wrong about that.
General Comment: As mentioned above, I was somewhat skeptical of Wikipedia as soon as I heard of it, but was willing to give it a chance. My skepticism soon turned into criticism as I began reading it, especially in areas in which I'm an expert.
How can you test a reference work or other source of information for reliability? Choose a topic about which you are already knowledgeable, preferably to the level of expertise, then see what the source has to say about it. So, I read Wikipedia entries on logic, including those on logical fallacies. Not only were some of these entries inaccurate, some didn't even make sense. In addition, many defenders of Wikipedia claim that errors are quickly corrected, whereas some of the mistakes that I noticed persisted for years or are still present7.
Sanger seems to have followed much the same path as I took, though more slowly. However, I get the impression from his more recent essays that he's now passed me in his skepticism, though that may be because I simply haven't paid much attention recently.
Finally, some comments about Sanger's writing: he's a philosopher who doesn't write like a philosopher writing for other philosophers. So, he spends little time referring to the views of famous philosophers or citing the philosophical literature in a way designed to impress other philosophers. This is not to say that his views on knowledge and expertise have been dumbed down, but that he writes about these topics so clearly that I think any intelligent person will be able to understand him.
Publication Date: 2020
Comment: This book is from last year, obviously, but I only found out about it this year.
The Blurbs: There are few blurbs for this book, and those it has are mainly descriptive of who Sanger is.
Disclosure & Disclaimer: I've never met Sanger in the flesh, but belonged to an email discussion list on systematic philosophy he ran back in the '90s. This is a newish book and I haven't read it yet, so can't review or recommend it, but its topic interests me and may also interest Fallacy Files readers.
- P. ix; all page number citations are to the new book.
- P. x.
- P. 6.
- Pp. x-xi.
- P. xi.
- I use the online version of the Encyclopedia Britannica, and only fall back on Wikipedia for information on popular culture that Britannica doesn't cover, and for which accuracy is not so important.
- As an example of an entry that contains novice errors, see: Wikipedia Watch, 10/22/2008. Interestingly, the "Talk" page for the Wikipedia entry discussed contains a good explanation of some of what's wrong with it, but the entry is still uncorrected after over a dozen years.
A Second Meeting of the New Logicians' Club
On this Independence Day, when you're through marching in a parade, eating hot dogs, and watching fireworks, here's a puzzle you can while away some of your free time on.
After attending my first meeting of the New Logicians' Club as a guest1, I decided to join. On the night of my first meeting as a member, the club was again playing the truth-tellers and liars game, which was the game where every member of the club was randomly assigned the role of either a truth-teller or a liar and required to answer every direct question accordingly.
Unfortunately, I arrived late for the meeting and, as a result, all the other members had already received their assignments as either truth-tellers or liars for the evening, so I didn't know who was what. Thankfully, I was assigned the role of truth-teller, and everything I tell you about that evening is true2.
The dinner had already started when I arrived and was seated at a round table with four other members of the club. I asked the member seated directly across from me―whose name was Arnauld, according to a tag on his lapel―whether he was a truth-teller or liar for the evening. He mumbled something inaudible to me because his mouth was full of food. Turning to the member seated next to him, whose name was apparently Bolzano, I asked what Arnauld had said.
"Arnauld said that he's not a liar", Bolzano replied. The member sitting next to me, whose name was Church, whispered to me: "But Arnauld was lying."
The fourth member at my table, named De Morgan, added: "Arnauld and Church are either both truth-tellers or both liars."
Finally, Arnauld swallowed his food and was able to speak audibly: "That's not true!" he blurted, glaring across the table at De Morgan.
How confusing! Can you help me determine which logicians were truth-tellers and which liars?
Extra Credit: What were the first names of those four logicians?
- Arnauld: truth-teller
- Bolzano: truth-teller
- Church: liar
- De Morgan: liar
Explanation: We saw in the first puzzle at the club1 that any member asked directly whether he or she is a truth-teller or a liar will always answer truth-teller and not a liar. Therefore, Bolzano was telling the truth when he said that Arnauld denied being a liar.
We also learned from the previous puzzle that a member of the club can only accuse another member of being a liar if the two members are opposites, that is, one is a truth-teller and one is a liar. Since Church accused Arnauld of lying, this means that they are opposites. However, De Morgan claimed that they were either both truth-tellers or both liars, so De Morgan is a liar. Finally, since Arnauld denied what De Morgan had just said, he must be a truth-teller, which makes Church a liar.
Extra Credit Solution: The four logicians were named: Antoine Arnauld, Bernard Bolzano, Alonzo Church, and Augustus De Morgan. If you were able to answer this question correctly, you are definitely New Logicians' Club material.
Disclaimer & Disclosure: Not everything I say is a lie, but this puzzle is. There is no New Logicians' Club, as far as I know, and even if there is I'm not a member.
- For the first meeting of the club, see: A Meeting of the New Logicians' Club, 5/30/2021
- Of course, if I were a liar, I would say the same thing!
Perpetrate or Perpetuate
Spell-checking programs are useful against some types of misspelling, but they won't catch everything. Nonetheless, a good one will catch most common misspellings, thus freeing you up to look for the rare ones. In particular, they won't catch a misspelling that just happens to spell a different word than that intended. For example, the word "led", which is the past tense of the verb "to lead", is often misspelled as "lead"1. Also, "parity" and "parody" are occasionally confused, and a spell checker probably won't notice2.
The words "perpetrate" and "perpetuate" are so similar in spelling, differing by only one letter, that they are difficult to distinguish at a glance. However, their meaning is very different: "to perpetrate" means to commit a crime, or some other bad action3. In contrast, "to perpetuate", means to make something perpetual, that is, to cause it to continue indefinitely4. Since both words are transitive verbs, it's unlikely that even a program that checks grammar will prevent you from confusing them. Given that both words are uncommon, you would expect that confusing them would be even less common, but it's common enough to have been warned against in at least two usage books5.
A little over a year ago, I noticed the following sentence in a professionally published book6: "One can even book a cabin on the annual Conspira-Sea Cruise, which allows passengers to not only heal from all of the conspiracies that have been perpetuated upon them but also watch for alien visitors in the night sky.7" The supposed conspiracies were allegedly perpetrated on the passengers, not perpetuated.
Recently, I came across the following confusing sentence in another book from a different professional publisher: "Guys like Omar helped bring fresh clean skins into the jihadi ranks and inspired those at home who were unable to get to places like Somalia, Yemen, Iraq, or Pakistan but were willing and able to perpetuate violence locally.8" With "perpetrate" substituted for "perpetuate", it's less confusing.
Please don't perpetuate the perpetration of this peccadillo.
Update, 7/19/2021: I was just doing some research on the Tamara Rand hoax of 1981 when I came across the following sentence in a statement by a television host who participated in it: "I have perpetuated a hoax on the public and feel very much ashamed.9" The hoax was perpetrated, not perpetuated, since it was quickly exposed. It's surprising to come across another example of this seemingly rare mistake in a little more than two weeks after writing the above entry. Also, in all three of these examples the mistake is in the same direction, namely, substituting the incorrect "perpetuate" for "perpetrate".
- See: Get the "Lead" Out, 2/5/2007.
- Parity or Parody, 3/18/2021.
- "Perpetrate", Cambridge Dictionary, accessed: 7/1/2021.
- "Perpetuate", Cambridge Dictionary, accessed: 7/1/2021.
- Bill Bryson, Bryson's Dictionary of Troublesome Words: A Writer's Guide to Getting it Right (2002)
- Harry Shaw, Dictionary of Problem Words and Expressions (Revised Edition, 1987)
- Conspiracy Theories: A Complete-Enough Picture, 6/17/2020.
- Joseph E. Uscinski, Conspiracy Theories: A Primer (Rowman & Littlefield, 2020), p. 5.
- Clint Watts, Messing With the Enemy: Surviving in a Social Media World of Hackers, Terrorists, Russians, and Fake News (Harper, 2018), p. 4; emphasis added.
- Myram Borders, "Hollywood psychic Tamara Rand's prediction of the attempted assassination…", United Press International, 4/5/1981.