WEBLOG

Previous Month | RSS/XML | Current | Next Month

July 31st, 2013 (Permalink)

Headline

San Diego mayor didn't get sexual harassment training, lawyer says

He learned by doing?

July 26th, 2013 (Permalink)

Check it Out

Why does so much research turn out to be wrong? Why is it that a study comes out claiming that coffee causes cancer, then a later report says that coffee prevents cancer? Statistician Andrew Gelman has an article focusing on a study that makes the rather ridiculous claim that women are more likely to wear red clothing when they are fertile, but this is just an example of what can go wrong, including in studies of more plausible hypotheses.

One reason, according to Gelman, is the multiple comparisons fallacy:

The standard in research practice is to report a result as “statistically significant” if its p-value is less than 0.05; that is, if there is less than a 1-in-20 chance that the observed pattern in the data would have occurred if there were really nothing going on in the population. But of course if you are running 20 or more comparisons (perhaps implicitly, via choices involved in including or excluding data, setting thresholds, and so on), it is not a surprise at all if some of them happen to reach this threshold.

One thing that Gelman doesn't mention, though it's implicit in his discussion of Bem, is that replication is not an optional part of the scientific method. The history of parapsychology is the history of studies like Bem's: every so often a new one comes along, there's excitement, and it's touted by proponents as proof of ESP; but no one can replicate it, eventually it's forgotten, and then another new study comes along―and the process begins anew. This is why parapsychology remains a pseudoscience and not the real thing.

I think that Gelman's last sentence is potentially misleading: "The system of scientific publication is set up to encourage publication of spurious findings." There are definitely some perverse incentives in academia, including the pressure to "publish or perish" which encourages quantity of publications over quality. So, Gelman could have more accurately written: The system of scientific publication is set up to encourage publication, including spurious findings.

Moreover, some of the mechanisms that should weed out the spurious findings don't work as well as they ought: Peer review tends to be a thankless job and, therefore, superficial. Journals are biased in favor of publishing positive results rather than negative ones, including failures of replication. As a result, there are far too few attempts made to replicate studies, and those that are made may not get published.

Moreover, we don't know how many studies fail to achieve statistically significant results and are simply filed away―"the file drawer effect". If nineteen studies showed no statistically significant relationship between red clothes and fertility, then there would be no surprise that a twentieth did so. Of course, it's unlikely that there are any other studies done of this particular hypothesis, but how many have been done on coffee and cancer?

Source: Andrew Gelman, "Too Good to Be True", Slate, 7/24/2013

July 23rd, 2013 (Permalink)

The History of "the Reverse Mussolini Fallacy"

I like The Volokh Conspiracy and frequently point readers to articles from it, but the conspirators sometimes step out of their areas of expertise in the law to moonlight as logicians or linguists. Many years ago, head conspirator Eugene Volokh named what he called "the Reverse Mussolini Fallacy" (RMF), and I pointed out at the time that this was actually a new name for the old fallacy of Guilt by Association (GbA). Now, conspirator Ilya Somin writes (see the Source, below): "One of senior Conspirator Eugene Volokh’s cleverest inventions is the Reverse Mussolini Fallacy."

Just to try again to set the record straight (see the Resource, below, for the first attempt), Volokh did not "invent" the fallacy of GbA, he just renamed it "the Reverse Mussolini Fallacy". That's fine, as far as it goes, though I don't like the name for reasons that I explained in the previous entry on this subject. However, that's a matter of taste, and others might like it, but to give something a new name―no matter how clever―is not to "invent" it.

The Fallacy Files entry for GbA goes back to the beginning of this website in 2001, at least four years before Volokh wrote his original entry, but even I didn't "invent" the fallacy. I'm not sure who was actually the first person to recognize it, but it wasn't Volokh unless there is a different, earlier source. The Source that I cited in the entry was the third edition of T. Edward Damer's textbook Attacking Faulty Reasoning, which was published in 1994, over ten years before Volokh's post on RMF. Here's how Damer defines it in his section on "Assigning Guilt by Association":

This fallacy involves the manipulation of negative feelings, by pointing out that the opposing view is held by people or groups we don't like or don't usually agree with. This appeal encourages one to accept the arguer's position in order to avoid any guilt by association with those held in such negative esteem. (P. 54)

This is highly general, but notice that it is a description of GbA as a logical fallacy rather than the common, confusing usage in which "guilt by association" refers to accusing or suspecting someone based on social relationships. In the logical sense, it is not GbA to accuse someone of being a fascist because that person associates with fascists. Rather, it would be GbA, in the logical sense, to reject a view just because it is held by some fascist, which is why rejecting punctual trains because of Mussolini is an example of GbA.

Michael Labossiere's work on fallacies, still available at The Nizkor Project, includes a page on the fallacy using the "Guilt by Association" name. The page (see the Source, below) goes back at least to 1997, but is copyrighted 1995. Here's Labossiere's second example, which is clearly the type of reasoning that Volokh was labelling "the Reverse Mussolini Fallacy":

Jen and Sandy are discussing the topic of welfare. Jen is fairly conservative politically but she has been an active opponent of racism. Sandy is extremely liberal politically.

Jen: "I was reading over some private studies of welfare and I think it would be better to have people work for their welfare. For example, people could pick up trash, put up signs, and maybe even do skilled labor that they are qualified for. This would probably make people feel better about themselves and it would get more out of our tax money."

Sandy: "I see. So, you want to have the poor people out on the streets picking up trash for their checks? Well, you know that is exactly the position David Count endorses."

Jen: "Who is he?"

Sandy: "I'm surprised you don't know him, seeing how alike you two are. He was a Grand Mooky Wizard for the Aryan Pure White League and is well known for his hatred of blacks and other minorities. With your views, you'd fit right in to his little racist club."

Jen: "So, I should reject my view just because I share it with some racist?"

Sandy: "Of course."

That's the best that I can do in describing the history of the fallacy, since neither Damer nor Labossiere supply citations to sources, and the trail goes cold around the mid-'90s. My guess is that there are earlier sources, but I haven't found any so far. If you know of an earlier reference, please let me know.

Sources:

Resource: Don't Name that Fallacy!, 12/2/2005

July 21st, 2013 (Permalink)

Book Review: Deadly Choices

Earlier this month (see Resource 2, below), in the course of pointing to his new book, I mentioned that I had read Paul Offit's previous book Deadly Choices. So, here's a brief review of it, a little late maybe but better than never:

Title: Deadly Choices
Subtitle: How the Anti-Vaccine Movement Threatens Us All
Author: Paul A. Offit
Date: 2010
Quote… In the early 1900s, children routinely suffered and died from diseases now easily prevented by vaccines. Americans could expect that every year diphtheria would kill twelve thousand people, mostly young children; rubella (German measles) would cause as many as twenty thousand babies to be born blind, deaf, or mentally disabled; polio would permanently paralyze fifteen thousand children and kill a thousand; and mumps would be a common cause of deafness. Because of vaccines, all these diseases have been completely or virtually eliminated. But now, because more and more parents are choosing not to vaccinate their children, some of these diseases are coming back. How did we get here? How did we come to believe that vaccines, rather than saving our lives, are something to fear? (P. xviii) …Unquote
Review: This is not a book about logical fallacies, but a book that reveals the consequences of fallacious thinking. As a result, it contains examples of fallacies and of their often deadly effects. If you've ever wondered where's the harm in the kind of pseudoscientific thinking and conspiracy theories that have led people to skip or delay vaccination, this book offers a case study. How did we get here? These are the main fallacies implicated:
  1. Post Hoc: It's no accident that the diseases that vaccines are blamed for are ones usually diagnosed in early childhood:
    The important question…was whether the vaccine could cause permanent harm, such as epilepsy and mental retardation. Answering this question isn't as easy as it seems. That's because every year in the United States, in England, and throughout the world, children suffer epilepsy and mental retardation; this has been true for centuries, well before the pertussis vaccine was invented. Also, symptoms of epilepsy and retardation often occur in the first year of life, the same time that children are receiving three doses of vaccine. Given the widespread use of pertussis vaccine, most children destined to develop seizures or mental retardation anyway would likely have received it, some within the previous twenty-four or forty-eight hours. So, the only way to figure out whether the vaccine was the problem was to study thousands of children who did or didn't get it. If the vaccine were responsible, the risk of epilepsy or retardation would be greater in the vaccinated group. (P. 28)

    Parents of children who have seizures, or are diagnosed as epileptic or autistic after being vaccinated, may feel convinced by this coincidence that the vaccine caused the illness, but you just can't tell that way. It's reasonable to form a hypothesis that the vaccine caused the disease, but that hypothesis must then be tested scientifically to establish causation. As Offit explains, the hypotheses that vaccines cause epilepsy or autism have been tested and have failed the tests.

  2. Appeal to Nature: The notion that it's unnatural to vaccinate people is one of the most common arguments against it. This is ironic since vaccination may be the most natural way of fighting disease, because it trains the body's own defenses to recognize and be able to fight infections. Here's a nineteenth-century anti-compulsory vaccination activist:
    "Stay then the hand of the vaccinator," he wrote. "…Let Britannia put her foot on this iniquitous destroying, death-producing interference with nature's laws and crush it out." (P. 109)

    And here's a twenty-first century one:

    [Barbara Loe] Fisher also argued that natural infection is better than immunization. "Experiencing infectious disease, … has been part of the human condition since man has walked the earth…. Why do vaccinologists insist on assuming that the human immune system is incapable of dealing with that experience? Or benefiting from it?" (P. 76)

    And man has been dying from many of those diseases since he has walked the earth, at least until quite recently thanks to vaccination. Of course, death is natural, too.

  3. Appeal to Celebrity: Offit discusses several celebrities who have contributed to the crusade against vaccination. Jenny McCarthy is the most infamous, together with her former boyfriend Jim Carrey. Now McCarthy is apparently going to be the new co-host of the silly TV show The View, which previously gave the slot to conspiracy theorist Rosie O'Donnell (see Resource 1, below), so it's not exactly an unprecedented move.

    In a telling anecdote about the media and celebrity, Offit recounts how McCarthy refused to appear on an episode of the television show The Doctors with a representative of the American Academy of Pediatrics (AAP):

    …[I]n The Doctors program [Travis] Stork revealed how, on the strength of McCarthy's star power, she had rigged the show.

    MCCARTHY: Go call the AAP…and see if they'll sit down with us and they'll say, 'No….'

    STORK: Let me just say this openly to everyone. You know, we wanted to have someone from the AAP here today, but you refused to allow them to come. So if you want to engage them in a debate, they would have been here.

    …[Jenny] McCarthy is a celebrity. It's her celebrity that has landed her on shows like Oprah and Larry King Live. And it's her celebrity that has enabled her to determine the guest list. (Pp. 156-157, 162)

    It's good that Stork called McCarthy on her dishonesty, but why did the show agree to have her on at all, let alone allow her to dictate who else would appear? Obviously, it's for the same reason that The View would offer her a co-hosting gig.

    In addition to McCarthy, there's the comedian, talk-show host, and alleged "skeptic" Bill Maher. If Maher is a skeptic, I'm McCarthy's new boyfriend. Maher is so ignorant that he thinks that vaccination means "stick[ing] a disease into your arm", yet he calls other people "idiots" for getting vaccinated (p. 165).

    Of course, there's no reason why former Playboy bunnies or comedians should know anything much about vaccination, but there's also no reason why we should pay any heed to their opinions about it.

Resources:

  1. Silly Celebrity, Too, 3/31/2007
  2. New Book: Do You Believe in Magic?, 7/10/2013
July 19th, 2013 (Permalink)

Blurb Watch: Blackfish

An ad in The New York Times for the new documentary Blackfish shows four stars from its review by Time Out New York, which uses a five-star system. Many potential moviegoers may not know this fact and think that it is the highest rating possible. We've seen this before many times (see the Previous Blurb Watches, below), and Carl Bialik even mentioned it in passing in his "The Numbers Guy" column on movie rating scales. Time Out NY used to use a six-star scale, but they dropped one star a few years ago for some reason.

In the same ad, there is the following blurb:

"STIRRING. A vital, convincing proponent for the greater protection and understanding of such evolved and majestic creatures."-LOS ANGELES TIMES

The blurb leaves something out of the full context of the quote and doesn't even bother to indicate the omissions with ellipses:

The stirring documentary "Blackfish" vividly tracks the thorny case of Tilikum, a 12,000-pound killer whale and longtime SeaWorld Orlando attraction responsible for three deaths over the course of several decades in captivity. … Since SeaWorld declined to comment here…"Blackfish," named after the Native American term for orcas, remains decidedly one-sided. But when that "side" is such a vital, convincing proponent for the greater protection and understanding of such evolved and majestic creatures, it can't help but win.

Obviously, the adwriter was also a bit one-sided.

Sources:

Previous Blurb Watches:

An example of a misleading 3D bar chart
July 11th, 2013 (Permalink)

Charts & Graphs: The 3D Bar Chart, Part 2

In Part 1 of these entries on how three-dimensional bar charts can be misleading (see Previous Entry 4, below), we saw that an overhead perspective given to such a chart can exaggerate the heights of its bars. In addition to rotating such a chart on its x-axis, it's also possible to rotate it on its y-axis in such a way as to make one side of the chart appear closer to the viewer than the other. See the chart to the right for an example.

A quick glance at this chart is likely to give the impression that the fourth bar is taller than the first, because the fourth appears closer than the first in the graph's perspective, and therefore looms larger. However, a closer look will show that both bars represent the same value: .5. As we've seen with other 3D graphs, the extra dimension adds no new information, so it can only be justified on the grounds that the additional dimension makes the graph prettier or more striking.

The chart shown is one created to be an example of this effect, rather than one captured "in the wild". I prefer, when possible, to use real-life examples, but in this case I was not able to find a good one. This is some evidence that this type of misleading chart may be uncommon, and probably less common than the types we've seen previously.

As I've mentioned in previous entries, there's no rule against making bar charts three-dimensional, or rotating them on the y-axis, but it's important to keep in mind the possibility of distorting the apparent relative sizes of the bars in doing so.

Source: "Misleading graphs", BBC

Previous Entries in this Series:

  1. The Gee-Whiz Line Graph, 3/21/2013
  2. The Gee-Whiz Bar Graph, 4/4/2013
  3. Three-Dimensional Pie, 5/5/2013
  4. The 3D Bar Chart, Part 1, 6/3/2013

Next Entry in this Series: The One-Dimensional Pictograph

July 10th, 2013 (Permalink)

New Book: Do You Believe in Magic?

Dr. Paul Offit, whom I've mentioned in passing a couple of times previously (see the Resources, below), has a new book out titled: Do You Believe in Magic? The Sense and Nonsense of Alternative Medicine. I've just laid hands on a copy, so I haven't read it yet, but I have read and recommend his previous book Deadly Choices, which is a good answer to the question: Where's the harm?

What can we expect from Offit's new book? A lot of the appeal of "alternative" medicine is based on the notion that it is somehow more "natural" than what it's alternative to, though it's hard to see what's so natural about coffee enemas or taking lots of vitamin pills. Unsurprisingly, Part II of the book is titled: "The Lure of All Things Natural".

Another thing that wouldn't be surprising is some silly celebrity sightings, which brings us to Part IV: "When the Stars Shine on Alternative Medicine", with chapters on Suzanne Somers and Jenny McCarthy. The mother of all silly celebrities, Oprah Winfrey, puts in an appearance on the first page of chapter one!

What other tricks do the advocates of alternative medicine have up their sleeves? You'll have to read the book to find out!

Resources:

Fallacies:

July 4th, 2013 (Permalink)

A Fireworks Puzzle

Four young friends―one is named David―went to a store to buy fireworks for the annual Fourth of July picnic celebration of their families―one of which has the last name Glenn. Each child was given enough money to buy three different types of fireworks, and each did so but no two bought exactly the same three. Their parents wisely limited their purchases to four types of fireworks: sparklers, snappers, whistlers, and black snakes. Brian and the Harris child, who isn't Amy or Clara, both bought sparklers and whistlers. The Edwards child, who isn't Amy, bought both whistlers and black snakes, but the Flynn child bought only one of these types of fireworks. Clara bought some sparklers. What is Clara's last name?

Update (7/17/2013): Just to be clear, the first names of the four children are, in alphabetical order: Amy, Brian, Clara, and David.

Previous Month | RSS/XML | Current | Next Month