Previous Month | RSS/XML | Current | Next Month

WEBLOG

April 29th, 2021 (Permalink)

Fact-Checking1: What is a fact?

The English word "fact" is ambiguous, having at least the two following meanings2:

  1. Situations or States-of-Affairs: According to philosopher and logician Bertrand Russell: "If I look up a railway timetable and find that there is a train to Edinburgh at 10 A.M., then, if the timetable is correct, there is an actual train, which is a 'fact'".3 So, the fact in question is the situation or state-of-affairs that there is a train to Edinburgh at 10 in the morning. I'll continue to use the word "fact" for this sense.
  2. True Statements or Claims: In another sense of "fact", the statement in the timetable is a fact if it is true, that is, it "states a fact", meaning that there is a "fact" in the previous sense that makes the claim true. In Russell's example, the claim in the railway guide is a "fact" if there is a state-of-affairs that makes it true, that is, there is indeed an actual train to Edinburgh at ten in the morning. So, a "fact" in this sense is a true claim about the world4.

Neither sense of "fact" is what fact-checkers check; in other words, "fact-checkers" don't really check facts. Rather, fact-checkers check claims to see whether they are true or false by checking them against the facts in sense 1. If a claim checks out as true, then it is a "fact" in sense 2.

A typical fact-checking task involves either reading or listening to a bunch of sentences, then checking what you read or hear. However, not all of the sentences will be claims that should be checked, so the fact-checker's first task is to select the claims from those sentences.

First, not all sentences are true or false, so not all are claims. For instance, questions and orders are sentences, but not claims. Consider the following sentences:

  1. "The door is locked." This sentence is true or false, so it's a claim.
  2. "Is the door locked?" This sentence is neither true nor false. It's a question, not a claim.
  3. "Lock the door!" This sentence is neither true nor false. It's a command, not a claim.

Second, not all claims state a situation or state-of-affairs that could be checked, but if a claim does state such a state-of-affairs, I'll call it a "factual claim". A factual claim states that a situation or state-of-affairs holds; if that situation is a fact, then the claim is true. So, if it is a fact that the door is locked, then the claim "The door is locked" is a true factual claim.

Suppose, however, that it is not the case that the door is locked; then, the factual claim "The door is locked" is false. So, "factual claim" shouldn't be confused with "true statement", as there are both true factual claims and false ones. So, by calling a claim "factual" I do not mean to suggest that it necessarily states a fact and is, thus, true. In fact, for every true factual claim there is a false one, namely, its negation. The negation of a factual claim is also a factual claim, and it can be checked in exactly the same way as the affirmative claim. So, "The door is unlocked" is just as much a factual claim as "The door is locked", and the same fact makes one true and the other false.

There are two important classes of claims that are not factual ones: opinions and value judgments. I expect to have more to say about these non-factual claims in later entries, but for now suffice it to say that they are true or false, but not factual. Thus, opinions and value judgments are not the sort of claim that fact-checkers should check.

Another type of statement that is not factual is one such as: "There is a china teapot in an orbit around the Sun between the Earth and Mars"5. This is clearly a claim, since it is either true or false that there is such a teapot. Admittedly, it is extremely unlikely that there is such an object, but it's not logically impossible, so it could be true.

While it may be possible at some point in the distant future to test whether there really is such a teapot, it's currently impossible to check. Since there's no point in wasting time trying to check an uncheckable claim, it's not a factual claim that should concern fact-checkers. Sometimes it may turn out that a factual claim cannot be checked, but we must at least have some idea about how to go about it, and it must be practical to do so, before even attempting it. The difference between a factual claim and an uncheckable claim is not black or white, but a scale with factual claims at one end, uncheckable claims at the other, and a gray area in between.

To sum up: what fact-checkers check are factual claims, which may be true or false, in order to find out which one they are. To do this, one comparesthe factual claim with the relevant states-of-affairs in the real world―that is, the facts―to see whether the claim states a fact.

In the next entry in this series, we'll start looking at how to check factual claims against the facts.

Notes:

  1. For earlier entries in this series on fact-checking, see:
    1. Why You Need to be Able to Check Facts, 9/8/2020
    2. Fact-checking Vs. Nit-picking, 10/20/2020
    3. Four Types of Misleading Quote, 11/27/2020
    4. News Sources Vs. Familiar Quotations, 12/4/2020
    5. Rules of Thumb, 1/2/2021
    6. A Case Study, 2/4/2021
    7. Reliable Sources, 3/2/2021
  2. See: Peter A. Angeles, Dictionary of Philosophy (1981), under "fact". Angeles gives six different meanings of "fact", the first two and last of which relate to the first meaning given, above, and the third corresponds to the second meaning.
  3. Bertrand Russell, Human Knowledge: Its Scope and Limits (1948), p. 143; cited in Russell's Dictionary of Mind, Matter and Morals, edited with an introduction by Lester E. Denonn (1952), under "fact". Russell goes on to write: "The statement in the timetable is itself a 'fact,' whether true or false, but it only states a fact if it is true, i.e., if there really is a train"; emphasis in the original. This is somewhat confusing, but the sentence in the timetable is itself a fact if there really is such a sentence.
  4. Monroe C. Beardsley uses "fact" in this sense, see his: Thinking Straight: A Guide for Readers & Writers (1950), p. 5. In an earlier weblog entry, I also used "fact" in this sense, see: Fact Vs. Opinion, 6/22/2018.
  5. This is an example, famous among philosophers, of a practically unfalsifiable claim created by Bertrand Russell in: "Is There a God?", 1952.
Poll Watch
April 27th, 2021 (Permalink)

What Biased Last Year's Polls?

The polls leading up to last year's election did about as good a job of predicting the electoral results as Nostradamus, which led some to declare political polling as "dead" as he is. As I mentioned soon after the election1, there was bound to be some subsequent examination of the polling leading up to the voting, given the widespread consternation over the underestimation of support for Republican candidates. Now, a group of Democratic pollsters have banded together to try to figure out what went wrong2.

The fundamental problem facing all survey research is getting a sample that is representative of the population. If such a sample is, indeed, representative of the population, then we are justified in generalizing from the part to the whole. In contrast, generalizing from an unrepresentative, or biased, sample is likely to lead to false conclusions.

The Democratic pollsters have put out a press release3 discussing some possible biases that may explain last year's polling failures:

  1. Turnout Error: People who don't vote are not going to affect the outcome of an election, so pollsters try to poll only those who are most likely to vote, that is, "likely voters". However, different pollsters may come up with different definitions of "likely voter", and who is likely to vote may change from election to election. Here's what the pollsters had to say about the possible effect of this error on last year's polls:
    Turnout is one of the hardest things for pollsters to account for. Simply asking people whether they will vote is unreliable, because people tend to overreport their likelihood of voting as it's socially desirable to do so. Instead, campaign pollsters use detailed databases of historical voting records for millions of individual voters to build a universe of what we believe will be the most likely electorate. This approach gets pollsters close to the mark most of the time but depends on the elections of the future looking like the elections of the past―and they often don't….

    Now that we have had time to review the voter files from 2020, we found our models consistently overestimated Democratic turnout relative to Republican turnout in a specific way. Among low propensity voters―people who we expect to vote rarely―the Republican share of the electorate exceeded expectations at four times the rate of the Democratic share. This turnout error meant, at least in some places, we again underestimated relative turnout among rural and white non-college voters, who are overrepresented among low propensity Republicans. …

    Unfortunately, turnout error does not explain the entirety of the error in 2020. … Some other source of error is in play.

    Here are some other possible sources of bias:

  2. Refusers: Some people don't answer the phone when a pollster calls, or refuse to take a poll when asked. The problem faced by pollsters is whether those who so refuse differ from the voting population in any other way relevant to the election. For instance, if Trump-voters were more suspicious of polls than Biden-voters, and thus more likely to refuse to answer, that might skew the poll results in Biden's favor.
  3. Last-Minute Deciders: Some voters make up their minds on who to vote for in the last few days before the election, at a time when they cannot be polled. If those late-deciders tended to decide in one direction much more than in the other, then the polls could end up biased. The pollsters say:
    This one was a popular theory after 2016, because exit polls and callback surveys suggested late-deciding voters broke overwhelmingly for Trump. In 2020, however, polls were incredibly stable, with hardly any undecided voters throughout the race, so this probably did not play a major role, at least at the presidential level.
  4. A COVID-19 Stay-at-Home Bias: More people were staying at home last year than usual due to the coronavirus epidemic, and thus some were available to answer the phone that otherwise might not have been. Did these new stay-at-homes change the composition of the samples in a way that might have biased the polls? Were Biden-voters more likely to stay home than Trump-voters?

Except for the last one, each of these sources of bias is possible in all political polling, so unless the supposed COVID-19 bias was the most important one, or the pollsters can figure out how to compensate for the others, we can expect future polls to be similarly biased.

Notes:

  1. Post Mortem, 11/11/2020.
  2. Steven Shepard, "Dem pollsters acknowledge 'major errors' in 2020 polling", Politico, 4/13/2021. The American Association for Public Opinion Research also has a task force studying the performance of the general election polls, and we should hear its results later this year; see: "AAPOR Convenes Task Force to Formally Examine Polling Performance During 2020 Presidential Election", American Association for Public Opinion Research, 2/13/2020.
  3. ALG Research, Garin-Hart-Yang Research Group, GBAO Strategies, Global Strategy Group & Normington Petts, "Revisiting Polling for 2021 and Beyond", Democracy Docket, 4/13/2021. All subsequent quotes are from this release.
April 14th, 2021 (Permalink)

New Book

Quote: "When you think of someone with excellent judgment, what traits come to mind? Maybe you think of things like intelligence, cleverness, courage, or patience. Those are all admirable virtues, but there's one trait that belongs at the top of the list that is so overlooked, it doesn't even have an official name. So I've given it one. I call it scout mindset: the motivation to see things as they are, not as you wish they were. Scout mindset is what allows you to recognize when you are wrong, to seek out your blind spots, to test your assumptions and change course."1

Title: The Scout Mindset

Comment: The title of this new book comes from a distinction that its author makes between two different "mindsets"2:

The Soldier Mindset The Scout Mindset
Reasoning is like defensive combat. Reasoning is like mapmaking.
Decide what to believe by asking either "Can I believe this?" or "Must I believe this?" depending on your motives. Decide what to believe by asking, "Is this true?"
Finding out you're wrong means suffering a defeat. Finding out you're wrong means revising your map.
Seek out evidence to fortify and defend your beliefs. Seek out evidence that will make your map more accurate.
Related concepts: Directionally motivated reasoning, rationalizing, denial, self-deception, wishful thinking Related concepts: Accuracy motivated reasoning, truth-seeking, discovery, objectivity, intellectual honesty

I haven't heard or used this "soldier vs. scout" analogy before, but I've been making similar points about reasoning since the beginning of this website. In particular, I've discussed the dangers of thinking of argumentation or debate as a type of conflict, as in the "soldier mindset" that reasoning is defensive combat3―it's not just defensive, though, since some reasoning is offensive4.

Another way to express this "soldier vs. scout mindset" analogy is that argumentation needn't be a zero-sum game―that is, a game in which one side wins at the expense of the other―but can be win-win. The "soldier mindset" thinks that to win is to defeat the enemy, and doesn't realize that it may be better to "lose". Finding out that you're wrong about something―that is, "losing" an argument―means finding out that your map is wrong and needs revising, which is important to know.

So, I agree with the points it's used to make, but I'm not too happy with the analogy itself, for several reasons. First, it wasn't immediately obvious to me, and needed explaining, what the difference between a soldier and scout was supposed to be, and how that relates to the issue of reasoning. More importantly, it's based on the usual conflict metaphor for argumentation, which I think is itself part of the problem. If you think of argumentation as competitive, you'll adopt the "soldier mindset"; whereas, if you think of it as co-operative, you can at least be more open to the "scout mindset".

In addition, a scout is a type of soldier, if I'm not mistaken, and I don't think that actual soldiers are more likely to exhibit their namesake mindset than any other group―such as lawyers or politicians, say. The characteristics of the mindset―rationalizing, denial, self-deception, wishful thinking―are terrible traits for soldiers to have. If you're a soldier, the "soldier mindset" could get you killed.

Now, I doubt that the author intended to suggest that real-life soldiers suffer from the mindset named for them more than other people. Rather, the underlying point is the distinction between reasoning as conflict and as co-operation. So, there's a lot wrong with the "soldier" versus "scout" analogy, but I don't have a better one to offer in its place.

Subtitle: Why Some People See Things Clearly and Others Don't

Comment: This is a rather black-or-white subtitle, which may be the fault of the publisher rather than the author. I would include black-or-white thinking as one of the characteristics of the "soldier mindset", that is, thinking that everyone is either an ally or an enemy to be treated accordingly. The alternative "scout mindset" is that you may be able to learn more from your supposed "enemy" than from your alleged "friends", and those "enemies" may tell you the truth when your "friends" won't.

As a subtitle, I would have preferred: "Why Some People See Things More Clearly Than Others Do". The subtitle connects with another problem of the soldier vs. scout dichotomy, namely, that it suggests that everyone falls into one of these two and only two categories. Probably no one is all one or the other, but we're all some mixture of the two, with some leaning in one direction and some in the other. What's needed is not a dichotomy, but a continuum from the "soldier" on one end to the "scout" at the other.

Author: Julia Galef

Comment: This appears to be her first book. I don't know much about her, though I have seen a few short videos that she's done related to the topics discussed in the book. The book's "About the Author" note mentions that she's a podcast host, but doesn't say anything about her background or education. The videos that I saw were fine, but a book is an entirely different proposition.

Date: 2021

The Blurbs: The book is positively blurbed by Annie Duke5 and Philip Tetlock6, among others, which is encouraging.

Disclaimer: This is a new book―in fact, it appears to have been released just yesterday―and I haven't read it yet, so can't review or recommend it. However, it looks interesting to me and may interest Fallacy Files readers.7

Notes:

  1. "Introduction"; paragraphing suppressed.
  2. You can see a graphic of this table on the Amazon page for the new book.
  3. See, for instance: Rule of Argumentation 2: Be ready to be wrong!, 1/26/2019
  4. I don't mean "offensive" in the sense of offending others, but in the sense of aiming to refute their views rather than just defending your own. As the saying goes, sometimes the best defense is a good offense.
  5. For Duke, see: New Book: Thinking in Bets, 3/30/2019
  6. For Tetlock, see: Check it Out, 3/26/2009
  7. Revised to fix some awkward writing, 4/18/2021.
Puzzle
April 5th, 2021 (Permalink)

A Chinese Family Problem

One unfortunate effect of the one-child policy in China was to produce an imbalance of men to women, because so many couples wanted a boy more than a girl. As a result, after several years of the policy there were more young men in China than young women, meaning that many young Chinese men would have to live without a wife.

In order to correct this imbalance, a government bureaucrat in the same agency that had devised the one-child policy proposed a solution. The policy would be revised to say that couples could have as many girls as they wished, but must stop having children after one boy.

"Clearly," argued the bureaucrat, "this revised policy will correct the imbalance between the sexes. There will be families of one boy, one girl and a boy, two girls and a boy, three girls and a boy, and so on. Since Chinese parents want most of all to have a boy, they will keep having babies until they get one, thus producing more girls than boys. In just a few years, the imbalance will disappear!"

Assuming that the Chinese people obey the new policy as written, will it redress the imbalance in a few years?

WRONG AGAIN DUMMY
April 2nd, 2021 (Permalink)

2020 Hindsight, Part 2

Last month, we looked at a "prediction" of the coronavirus epidemic attributed to Nostradamus, which turned out to be a retrodiction created after the epidemic had got underway1. This month, let's look at some predictions of the events of last year attributed to Nostradamus that were published before the year started. How accurate were such predictions, and did the famed seer foresee the most important event of the year, namely, the coronavirus epidemic?

On the last day of 2019, the British tabloid newspaper Express published an article with the five top predictions for 2020 from the Nostradamus2020.com website2. Here they are in the order given by the article:

  1. Trump wins re-election: I suppose that some of Trump's supporters, and perhaps even the man himself, would claim that he really did win so that the prediction came true. Had Trump read and believed this when he claimed that he won? Of the five prognostications, this is the only one for which the newspaper article supplies a quatrain:
    The exiles because of anger and intestine hatred
    Will bring a great conspiracy to bear against the King
    Secretly they will place enemies as a threat,
    And his own old ones against them sedition.3

    This is a loose translation of Nostradamus' quatrain I-13:

    Les exilés par ire, haine intestine,
    Feront au Roi grand conjuration:
    Secret mettront ennemis par la mine,
    Et ses vieux siens contre eux sédition.4

    Of course, this quatrain doesn't predict anything about Trump or anyone else being re-elected. Instead, it predicts a plot against some unnamed King. Maybe Trump would have liked to be king, but he was only president. Nostradamus lived during the time period when there were still kings with real power, so there's no reason to think that he was using the word figuratively. Other than that, there's no indication of when the prediction is supposed to come true, nor any details that apply to Trump rather than some real king. Now, if it had predicted a conspiracy against the "great orange man", I would be impressed.

    Furthermore, at the time that this article was published, there was already a conspiracy aimed at impeaching Trump and removing him from the presidency that had been going on since before his election. So, even assuming that "the King" refers to Trump and not Elvis Presley, this is not a prediction, but an attempt to apply the quatrain to an event that was ongoing.

    A test of such a vaguely-written prophecy is how it was interpreted prior to the supposed event that fulfilled it. In this case, an early Nostradamus interpreter named Étienne Jaubert5, interpreted it as referring to the conspiracy of Amboise, which occurred in 15606. While Nostradamus lived until 1566, this quatrain is the thirteenth in the first book of "centuries"―collections of a hundred quatrains―so it was probably printed in the first edition of 1555, five years prior to the conspiracy.

    Another interpreter, Henry C. Roberts, wrote the following about this quatrain: "The progress of the Italian conspiracy is indicated―the rise of a fifth column rife with internal dissensions, culminating with a plot against King Victor Emmanuel.7" This is itself ambiguous since there were three kings of that name, but given that Roberts interpreted the two previous quatrains to be about the rise of fascism in Italy, he may have been referring to the third one8.

    In any case, the poem is so vaguely worded that it can be interpreted to apply to almost any plot against a king, but not to the re-election of President Trump.

  2. Earthquakes: There are earthquakes every year somewhere, but the website made a surprisingly precise claim:
    The prophet announced the occurrence of a strong earthquake in the future that will lead to the sinking of the entire state of California under the ocean's water. Nostradamus foretold a great earthquake in 2020, in America. In the New World (the West Lands), California seems to be the logical guess. This will happen next time Mercury is retrograde in Cancer (between June 18 and July 12).2

    Though its politicians seem intent on destroying it, the last I heard California was still there. How the website interpreters came up with such a precise prediction, I've no idea. Usually, Nostradamus had more sense than to make falsifiable prophecies. Fortunately for Californians, the Nostradamus interpreters were wrong again.

  3. The United Kingdom will gain a king. The last I checked, it still has a queen.
  4. Major war: Whatever you can say against 2020, you can't say this.
  5. Kim Jong-Un gone: Unfortunately, the dictator is still dictating.

So, that's zero for five, and no mention of an epidemic. Of course, these predictions are only Nostradamus as interpreted by his modern followers. This is convenient for him, since he's never to blame for a failed prediction; instead, it's the fault of whoever misinterpreted his vague and ambiguous doggerel.

Notes:

  1. 2020 Hindsight, 3/10/2021.
  2. Ciaran McGrath, "Trump re-elected, Kim Jong-Un out―& King William? Nostradamus predictions 2020 Revealed", Express, 12/31/2019. Unsurprisingly, the website is no longer available, but you can see the Wayback Machine's cache of its main page here: "Nostradamus Predictions for 2020", Internet Archive, 12/8/2019.
  3. This is Edgar Leoni's translation, see: Nostradamus and His Prophecies (1982), p. 135.
  4. See: Leoni, op. cit., p. 134.
  5. For Jaubert, see: Leoni, op. cit., pp. 62-63. Leoni finds this interpretation credible, but adds: "The wording, however, is sufficiently general that the quatrain may fit other palace plots of history as well. (p. 572)" The aptly-named Erika Cheetham also adopted Jaubert's interpretation, see: The Prophecies of Nostradamus, translated, edited & introduced by Erika Cheetham (1981). She translated the quatrain as follows:
    Through anger and internal hatreds,
    The exiles will hatch a great plot against the king.
    Secretly they will place enemies as a threat,
    And his own old (adherents) will find sedition against them.
  6. Editors, "Conspiracy of Amboise", Encyclopaedia Britannica, accessed: 3/12/2021.
  7. The Complete Prophecies of Nostradamus, translated, edited, and interpreted by Henry C. Roberts (New Revised Edition, 1982).
  8. Editors, "Victor Emmanuel III", Encyclopaedia Britannica, accessed: 3/12/2021.

Previous Month | RSS/XML | Current | Next Month | Top of Page