Skip to content

Are terrorists statistically significant?

tsa_securityHere’s another discussion relevant to the earlier post that DO SOMETHING is not a helpful government response to the current terrorismscare:

[T]he key point about identifying al-Qaeda operatives is that there are extremely few al-Qaeda operatives so (by Bayes’ theorem) any method you employ of identifying al-Qaeda operatives is going to mostly reveal false positives.

(From Matthew Yglesias via Tyler Cowen ONCE AGAIN, I think I’m now Tyler’s full-time RA).

How does this relate this to our usual statistical analysis? Proving someone is a terrorist is analogous to proving a nonzero effect that confirms an economic theory. We allow a rate of false positives of 5 percent (“statistical significance at the 5 percent level”) for showing that, say, good institutions have a positive effect on development. The false positives do not automatically swamp the true positives, because the true effects of  one thing on something else are not as rare as terrorists.

To have a low rate of false positives, we have to accept a high rate of false negatives. But we don’t care about false negatives. You failed to show an effect of your favorite magic ingredient X on development? Too bad, the burden of proof is on YOU if you want to add your ridiculous theory to the existing development knowledge.

Contrast airport security, where we DO care about false negatives (i.e. failing to detect a terrorist). To reduce false negatives even more (as everybody is demanding ), we would have to accept MORE false positives. This would swamp even more the rare genuine terrorists.

Yglesias used a hypothetical rate of false positives of 0.1 percent in his discussion of screening 15 million British Moslems. Of course, TSA makes it much worse by screening each and every of the 800 million airline passengers annually in the US — including my 80 year old mother whose only suspicious behavior is hiding her handbag in fear of NYC purse snatchers. A false positive rate of 0.1 percent times 800 million means that false positives would be 800,000 people.

Have you seen 800,000 terrorist suspects milling around at airport security? No, I haven’t either. So the true TSA false positive rate must be even lower, which must mean the false negative rate must be a lot higher than the TSA would like to admit (as confirmed by audits). Intensified universal screening cannot possibly work: QED. (For useful alternatives, consult the people in the know.)

As Shakespeare once said about TSA:

It is a tale told by an idiot, full of sound and fury signifying nothing.

OBLIGATORY AID PARALLEL TO TODAY’S APPARENTLY UNRELATED NEWSWORTHY TOPIC: the Do Something approach in aid has not been a great success either. Although it is still popular in aid and social activism, as illustrated by the nearly 300,000 followers on Twitter of @DoSomething, who wrote the following “Tweet”:

Its really easy to be a critic. Its really hard to be a do-er who actually makes stuff happen.

Stuff happen like click on a non-binding poll on their web site whether unnamed state legislators who don’t check web sites should pass laws against texting while driving. That may be easier than being a critic.

UPDATE: announcement today that TSA will piss off 14 mostly Muslim countries by subjecting fliers from those countries to the US to universal invasive screening. Thank goodness the terrorists are so dumb they would never think of flying from ANOTHER country besides these 14!

This entry was posted in Cognitive biases, In the news and tagged , , . Bookmark the permalink. Follow any comments here with the RSS feed for this post. Both comments and trackbacks are currently closed.


  1. John wrote:

    “If to do were as easy as to know what were good to do . . .”
    Merchant of Venice

    Intensified screening may accomplish nothing of its putative purpose, but it’s designed above all else to signify. No soccer fan wants to see penalty kicks defended by statistically-suave goalkeepers taking the do-nothing approach, even if it’s just as effective as dramatic dives (I think this was one of the original examples of do-somethingism). Likewise, the strutting and fretting of the TSA is all about signifying to, rather than protecting, the public.

    Nothing can come of nothing. So rather than face a certain nothing, it’s almost more rational to try a dubious something even if we believe that the chances of it making things better are small.

    Posted January 5, 2010 at 1:05 am | Permalink
  2. Jim wrote:

    Forget the father for a moment. Here is the profile we already had on ‘crotch bomber’:
    1. He was from a high risk country.
    2. He was a Muslim.
    3. He paid cash for his ticket.
    First, before December 25, 2009 (which is after 9.11), I would have guessed that those three computer generated statistics would have automatically spit out a directive in Amsterdam for security there to have a look at him, including an interview and search. No human intervention needed. And if those three statistics do not constitute a bar high enough for systemic attention before getting on an airplane to or in the USA, then I naturally want to know what does.

    Now we find out that the system isn’t even automated in that way. So now I’m confused about what exactly we’re doing on the systems side. Because it doesn’t seem very efficient. Computers love to do that kind of stuff.

    Second, if some blond haired girls with pig tails were inexplicably going crazy on airplanes, would we be obtaining health certificates for the whole nation? I think not. So then I am forced to believe we as citizens are being given some teachable moment here by the elites regarding terrorists. I believe the lesson we are being taught is that Muslims and terrorists are not an identical set. I also believe that virtually all citizens realize that.

    So just like the boy who gets suspended for taking his Boy Scout knife to school since all knives are supposedly equal, we are actually being taught not to discriminate. That doesn’t make us non-judgmental. That doesn’t make us less racially prejudiced. That makes us stupid.

    Yglesias is being specious with his stats. The question is not whether to check ALL Muslims. The challenge is to be intelligent. The Israelis already do it, and spend far less money doing it while inconvencing far less people. It’s really not that hard.

    Posted January 5, 2010 at 1:07 am | Permalink
  3. Doing Something wrote:

    Instead of polling people it’s better to write a blog legislaters don’t read but that will through 2nd and 3rd hand sources convince a bunch of people foreign aid is a waste of money and Bono ate their children for lunch.

    Which is sad because the real message was that aid is only possible when you “just see it” on the ground.

    Posted January 5, 2010 at 2:19 am | Permalink
  4. William Easterly wrote:

    Dear Doing Something:

    suggested alternate response to this post: criticism of was unwarranted because we are really doing x,y, and z proven things that work.

    Posted January 5, 2010 at 8:08 am | Permalink
  5. joe wrote:

    Seems to me this is all nonsense anyway. If the terrorists really wanted to do some damage, you’d think there would be much easier targets than blowing up aeroplanes. Unless someone is planning on full body screening everyone going to a shopping mall or sports game one would think.

    On the other point, we live in a culture where ‘clicking’ or ‘retweeting’ is considered to be activism. Sadly, it just isn’t.

    Posted January 5, 2010 at 9:20 am | Permalink
  6. Miriam wrote:

    There definitely seems to be diminishing returns when fighting terrorists. You can put x number of dollars or hours into “fighting” terrorism, and could triple or quadruple that and probably not see much change in the result. But we quadruple that number anyway because we need to be accountable in the case of an attack. We need to be able to say we did “everything” we could and shift the blame to something else, even though, as this post suggests, there may be nothing we can do. But doing nothing doesn’t get politicians elected.

    Posted January 5, 2010 at 9:44 am | Permalink
  7. Manuel wrote:

    Hell, prof. Easterly, this is really an ill-thought take. After all, we all (except some people with suicidal tendencies) choose to “do something”, i.e., to live, when there is no obvious point in doing it (besides precisely doing it). The real question here is about choosing to do something when we know (or we should know) there is a definite better alternative. Yes, like screening airport passengers when you should know there are better alternatives to avoid terrorists to get on board. I just don’t get what all this fuss about “doing something” is about.

    Posted January 5, 2010 at 6:17 pm | Permalink
  8. Lisa wrote:

    I don’t think there’s any need to bring the organization Do Something into this conversation–it’s a nonprofit that exists to help teenage entrepreneurs with an idea for a way to better their communities get the startup capital they’d need to do so. Much like microfinance for good intentions.
    They receive thousands of grant applications and give out varying amounts of money to groups that are trying to better the US in a variety of ways (mostly that have nothing to do with international aid/foreign development) like doing food drives, sending care packages to soldiers, raising awareness about domestic abuse, etc.
    The name reflects the idea that teens can find their own ways to get more involved in their communities, not that we should be blindly trying any method possible to increase the standard of living in other countries. From what I can see, the group ought to have a more comprehensive guide to helping teens measure their impact and using said metric internally to decide whether to fund similar projects in the future, but the organization itself has very little to do with aid/your area of expertise.

    Posted January 5, 2010 at 10:22 pm | Permalink

3 Trackbacks

  1. By Tuesday Highlights | Pseudo-Polymath on January 5, 2010 at 8:59 am

    […] those statistics mostly make sense if you consider that a grandmother from Iowa is just as big a threat as a foreign national from the […]

  2. […] those statistics mostly make sense if you consider that a grandmother from Iowa is just as big a threat as a foreign national from the […]

  3. By uberVU - social comments on January 5, 2010 at 10:25 am

    Social comments and analytics for this post…

    This post was mentioned on Twitter by Rubin: Are terrorists statistically significant?

  • About Aid Watch

    The Aid Watch blog is a project of New York University's Development Research Institute (DRI). This blog is principally written by William Easterly, author of "The Elusive Quest for Growth: Economists' Adventures and Misadventures in the Tropics" and "The White Man's Burden: Why the West's Efforts to Aid the Rest Have Done So Much Ill and So Little Good," and Professor of Economics at NYU. It is co-written by Laura Freschi and by occasional guest bloggers. Our work is based on the idea that more aid will reach the poor the more people are watching aid.

    "Conscience is the inner voice that warns us somebody may be looking." - H.L. Mencken

  • Archives