Previous Month | RSS/XML | Current | Next Month


April 24th, 2015 (Permalink)

Wiki Watchee: The Persistence of Misinformation

One of the claims used in defense of the accuracy of Wikipedia is that misinformation inserted into the online "encyclopedia" is usually found and removed quickly, even in a matter of minutes. I've argued previously that this is an unjustified claim because the examples that can be pointed to are only those that have been found, which means that the sample we have is affected by survival bias―see the Wikipedia Watch for 12/21/2014, below.

Moreover, there is at least one known hoax that lasted nearly a decade before being discovered―see the watch for 3/16/2014, below. I suggested that what is needed to reliably evaluate Wikipedia for reliability is a systematic study of randomly selected articles by appropriate experts.

Now, a study of Wikipedia's accuracy has been conducted, though not the kind of general evaluation that I suggested. Nonetheless, it's a clever approach to studying the specific question of how rapidly misinformation gets corrected―see the Sources, below. What Gregory Kohs did was to insert pieces of misinformation into various Wikipedia articles and keep track of them. Unfortunately, the length of the experiment, which is measured only in months, cannot tell us how long such misinformation might last. However, it does show that the notion that most misinformation will be speedily fixed is incorrect, at least if one measures speed in terms of weeks. Many claims about the rapid repair of incorrect claims put it in terms of minutes, rather than weeks or even months, as shown by some of the quotes cited by Kohs. Yet, about two-thirds of Kohs' misleading edits lasted weeks, and half remained in place by the end of the experiment, more than two months later.

The most amusing, and at the same time sad, aspect of this experiment is what happened when it ended and Kohs tried to correct the remaining misinformation himself:

The second craziest thing of all may be that when I sought to roll back the damage I had caused Wikipedia, after fixing eight of the thirty articles, my User account was blocked by a site administrator. The most bizarre thing is what happened next: another editor set himself to work restoring the falsehoods, following the theory that a blocked editorís edits must be reverted on sight.


Previous Wikipedia Watches: 12/21/2014, 3/16/2014

April 23rd, 2015 (Permalink)


Do doctors understand test results?

An old rule of journalism goes: headlines that end with a question mark can safely be answered "no". So, it probably won't come as a surprise to you that the article with the above headline gives evidence that doctors often do not understand test results―see the Source, below.

Of course, I would hope that if doctors were readers of this website they would understand the results of medical tests better, since we've examined all of the mistakes discussed in the article:

Check it out.

Source: William Kremer, "Do doctors understand test results?", BBC News, 7/7/2014

Acknowledgment: Thanks to Lawrence Mayes for calling this article to my attention.

April 19th, 2015 (Permalink)

Rolling Stone's Worst-Case Scenario

If you had no idea things were that bad, they probably aren't.―Joel Best

Columbia University's Graduate School of Journalism recently released its report on the now retracted Rolling Stone (RS) magazine story about an alleged campus gang rape―see Source 3, below. It's an important and interesting case study of how journalism can go wrong. The report itself is long, but well worth reading. There are also a number of shorter but excellent commentaries―see Sources 4 through 6, below.

The report claims that confirmation bias played a role in what went wrong:

The problem of confirmation bias―the tendency of people to be trapped by pre-existing assumptions and to select facts that support their own views while overlooking contradictory ones―is a well-established finding of social science. It seems to have been a factor here.

I like Megan McArdle's description of "classic" confirmation bias―see Source 5, below:

Classic confirmation bias means that you ask questions that would confirm your theory, rather than ones that would disconfirm it. Say I give you a set of numbers in a set: 2, 4, 6, 8. Now, I say, tell me what the rules for inclusion in this set are. You can ask me a number, and I'll tell you whether it's in the set. Almost invariably, the next numbers people suggest are "10" and "12," and when you agree they're in the set, they proudly announce that the set is "even numbers." False: The set is "all positive integers." Why did they fail? Because they only suggested numbers that would confirm their theory, which also happen to be in the set. What they didn't do is suggest an odd number to see if it might also qualify.

While it's a good thing that people are becoming aware of confirmation bias, I'm not so sure that it played much of a role in this case. Instead, the reporter, editor, and fact-checker seem not to have insisted even on finding evidence in support of the accusation, let alone against it. They simply seem to have accepted the accuser's account, and in lieu of seeking out evidence to support or refute it, the reporter wrote and her editor edited the story in such a way as to conceal that it was based entirely upon one woman's accusations. As Jean Kaufman writes―see Source 4, below:

[RS] appear[s] to have jettisoned those time-honored procedures [of journalism] for reasons that were most likely both ideological and self-serving: the story was a perfect fit for their pre-existing biases about campus rape and its perpetrators, and the tale was so sensational that it could practically guarantee them a record number of readers. In other words, it was far too good to fact check. …Rolling Stone set out to find a particular type of narrative and [it] got a sensational one. They then were willing to suspend the journalistic standards they profess to hold dear in order to protect it from too many questions. Thatís not journalism, itís activism. For reporters, the greater their initial bias in one direction or other, more care must be taken to overcome it with more due diligence, not less….

McArdle appears to agree:

What I see when I read through the…report is the story of journalists who had an incredible story, one that would get them readers and professional acclaim, and, perhaps most important, give them the opportunity to right a great wrong. Their excitement about the story, their determination to tell it, blinded them to the problems, so that the old joke about a story being "too good to check" actually came true, with terrible consequences. And that should be a lesson to every journalist out there: The better your story, the harder you need to work to disconfirm it. Because the odds are, your brain is sending you all the wrong signals. Of course, it's not exactly news that our emotions can mislead us. That's why we have professional rules, such as "always contact the other side for comment," in the first place. Rolling Stone got taken by a fabulist. But it was not the victim of fraud; it was a co-conspirator in self-deception.

There is one factor that I think played a role in this debacle that's not mentioned in the Columbia report. Moreover, it does not involve a failure to live up to journalistic standards, but instead a standard practice in journalism, namely, that of searching for a dramatic anecdote to build a story around. Jay Rosen is the only commenter on the case that I've noticed mention this problem―see Source 6, below:

The most consequential decision Rolling Stone made was made at the beginning: to settle on a narrative and go in search of the story that would work just right for that narrative. The key term is emblematic. The report has too little to say about that fateful decision, probably because itís not a breach of procedure but standard procedure in magazine-style journalism. (Should it be?) This is my primary criticism of the Columbia report: it has too little to say about the ďemblem of…Ē problem. [Ellipsis in the original.]

Initially, RS's article was supposed to be about the general problem of rape on university campuses and how administrations tend to deal with it. The reporter then set out to find the most dramatic case she could to illustrate this general problem, and presumably settled on the gang rape story because it was the most extreme and horrifying one she found. However, there are problems with this approach to reporting:


  1. "Carl Sagan on Alien Abduction", NOVA, 2/27/1996
  2. Joel Best, Stat-Spotting: A Field Guide to Identifying Dubious Data (2008), pp. 11, 111-113
  3. Sheila Coronel, Steve Coll & Derek Kravitz, "Rolling Stone and UVA: The Columbia University Graduate School of Journalism Report", Rolling Stone, 4/5/2015
  4. Jean Kaufman, "Too Good to Fact Check", PJ Media, 4/7/2015
  5. Megan McArdle, "Rolling Stone Can't Even Apologize Right", Bloomberg View, 4/6/2015
  6. Jay Rosen, "Rolling Stoneís ĎA Rape on Campus.í Notes and comment on Columbia J-schoolís investigation.", Press Think, 4/6/2015

Fallacy: The Anecdotal Fallacy

April 17th, 2015 (Permalink)

New Version: Statistics Done Wrong

Alex Reinhart's Statistics Done Wrong, which was formerly only a website, is now a book in various formats, including paper! The new book is claimed to be three times the size of the web version. Unlike such books on statistics aimed at a general audience as Darrell Huff's and Joel Best's, it's not about the kind of statistical errors made by journalists reporting on scientific studies, or by advertisers or advocates misreporting them. Rather, it describes the mistakes that scientists themselves make, and that lead to so many false and conflicting results. Reinhart discusses the following statistical mistakes that we've met here previously: the base rate fallacy, the multiple comparisons fallacy, and the regression fallacy. I haven't read the new book, but the web version is very clearly written, and has a minimum of actual math if that sort of thing scares you. So, this is not an introduction to statistics, but it does what such introductions don't do, which is explain the logic and illogic of statistics in a way that even non-mathematicians can understand.

Source: Alex Reinhart, Statistics Done Wrong

April 15th, 2015 (Permalink)

Puzzle it Out

If you haven't racked your brain enough doing taxes, there's a clever puzzle making the rounds that you might be interested in. It's being called a "math" puzzle, perhaps because it was a problem in a math olympiad for high school students in Singapore. However, it's really just a logic puzzle, since no mathematics is required to solve it. The original version of the puzzle was controversial enough that The New York Times published an article about it―see the Source, below. The controversy seems to have been at least partly due to the original wording of the puzzle, which was ungrammatical and unclear because it was presumably written or translated by someone who was not a native speaker of English. Revised and unambiguous wording of the puzzle is given in the Times article, and the solution is also clearly explained. Check it out.

Source: Kenneth Chang, "A Math Problem From Singapore Goes Viral: When Is Cherylís Birthday?", The New York Times, 4/14/2015

April 6th, 2015 (Permalink)

The Logical Problem of Evil

And God saw every thing that he had made, and, behold, it was very good.
And the evening and the morning were the sixth day.―Genesis 1:31, KJV

Chris Cox sends the following story that I expect many readers can sympathize with:

When I was in the 6th grade in Catholic school we learned about Lucifer and how he was God's most brilliant angel. And then, through pride, he fell from God's grace and became Satan and was cast into Hell. Thence he has tempted man to sin so he can gather their souls into Hell and deny them heaven.

One day a Monsignor came around to ask us questions about what we had learned and to allow us to ask him questions. I had been thinking about how God was all-powerful and how he was all-good, as taught in the first two pages of our catechism. So I asked the Monsignor why God didn't just snap his fingers and make the Devil disappear. As I remember he paused for a moment then said, "Well, there are some things God does without our understanding them. These things will be revealed to us when we join him in heaven." Or words to that effect.

Over the last thirty years or so I had been thinking of that classroom and why God would create Satan, Hell and all the other suffering, all because of Adam and Eve disobeying him (not to mention the fact he knew they were going to disobey him, and all the mental entanglements that gets you into). I also had been reading several books and articles in various publications. After some time I came to the conclusion that there was no god.

The argument I use comes from the first two or three pages of that first St. Joseph's catechism. In those pages we were taught that God was all-powerful, all-good, and all-knowing. Which brings me, finally, to my argument from evil that a perfectly good god can't create a universe with evil in it. Or as Paul Kurtz asked in his publication, Free Inquiry, "Why doesn't God abolish evil?

  1. He can't, and is therefore not Omnipotent, or
  2. He won't, and is therefore not Omnibeneficent."

My question concerns any fallacies in all of this: Is it sound?

You're raising a difficult problem that philosophers have written whole books about, but I'm not going to. In order not to write a whole book about it, I'll pass quickly over some complexities and carefully avoid distracting side issues. For lengthier but not book-length treatments, see the Sources, below. Also, as a logician and not a theologian or philosopher of religion, I will concentrate on a few logical points raised by your account:


Resource: Anthony Gottlieb, "Candide and Leibnizís garden", Voltaire Foundation, 2/3/2015. A brief discussion of the relationship between Candide and Leibniz. Contains some untranslated French.

Update (4/10/2015): A reader wrote to offer a version of the "free will" solution to the problem of evil, which I mentioned in the note to Source 1, above. This was one of the side issues that I was trying to avoid, but perhaps it's not obvious that it is a version of the "best of all possible worlds" defense.

The basic idea of the free will argument is that God created a world in which we have free will and that means we are free to do evil. With respect to free will and evil, there are four types of possible world that God could choose from:

  1. There is free will and there is evil. This seems to be the type of world we live in.
  2. There is free will but there is no evil. Some will claim that this type of world is not really possible, since if people have free will then it's possible that they will commit evil. However, free will does not necessitate that we commit evil, for if it did then in what sense would we be free? Therefore, it's possible that God could have created a world in which we have free will but have freely chosen not to commit evil. However, many philosophers have believed―wrongly in my opinion―that it is impossible for God to create people with free will who commit no evil, so let's put this possibility to one side.
  3. There is no free will but there is evil. This might also seem, at first glance, not to be possible but there is what's called "natural" evil, which is the evil resulting from earthquakes, volcanic eruptions, hurricanes, diseases, and so on. Thus, it's possible to have a world in which there is no "moral" evil―that is, the evil done by people with free will―but still have natural evil. However, it's arguable that a world in which we lacked free will would be one in which natural evil did not matter to us, since we would then be like "robots". For this reason, let's also put this possibility aside.
  4. There is no free will and no evil.

So, ignoring possibilities 2 and 3, the choice that God faced was between worlds of type 1 or 4, that is, between a world in which there was free will and evil (1) or one in which there is no free will and no evil (4).

Now, why would God choose 1 over 4? Presumably, because free will is of such great value that a world with both free will and evil is better than one with no free will but no evil. If that were not the case, then God would have chosen to create a worse world than He could have. Why would He do this? Only because he either could not help it, did not know any better, or did not wish to create the best world he could; in other words, only if he is not omnipotent, not omniscient, or not omnibenevolent.

Therefore, if God exists and is all three "omni"s, then Leibniz was right that this is the best of all possible worlds.

Source: Tim Holt, "The Free Will Defence", Philosophy of Religion, 2008

Previous Month | RSS/XML | Current | Next Month