Failing badly: Why biometrics and datamining isn’t going to save us from terrorists.

For obvious reasons, security, especially airplane security, is back in everyone’s thoughts and we had a lively discussion here recently about what methods would work.

While editing the wikipedia article about El Al, I found this fascinating article in Atlantic Monthly (pdf) on the web, which, while from 2002, is extremely current. The subject of the article, Bruce Schneier, a computer crypto geek and security consultant, explains why most of the approaches to increased secuity since 9/11 fail badly – that is, once they fail, they fail catastrophically. One example: all the additional biometric identification proposed now for our ID cards would not have prevented Mohammed Atta from flying his plane into the twin towers; he and his compatriots used their real identities. Similar things hold true for the tech-intensive attempts to keep bombs and weapons out of the plane. Once the attacker is beyond the barrier with them, the system fails badly.

A system that fails well, he holds, is instead one that does not depend on the attacker being unaware of it. He cites the awareness that, nowadays, passengers would risk their own life to tackle a hijacker. Knowing about this fact does not help the attacker, but knowing that old-style scanner cannot detect liquid explosives almost did. Armored cockpit doors meanwhile are a classic example of compartmentalization—they contain the failure after it has occured. And as the article nicely points out, methods like containment do not cause the widespread damage to civil liberties the way the data gathering approach does.

The third element he notes is the human one, where we come back to how I found the article. El Al does not (only) protect its planes by bomb sniffers and background checks. They talk to you before you board their planes.

15 thoughts on “Failing badly: Why biometrics and datamining isn’t going to save us from terrorists.

  1. Well, automation will always have a far larger margin of error and chance of failure than actual human testing.  I do software testing for a living, and there it definitely rings true.  We use both automation and human testing, and honestly, the automation is fairly limited.  Since it can’t adapt and change, you have to tediously code every possibility into it to make sure it is efficient enough to be effective.  However, there are always issues that you can’t forsee, that a live, thinking human will see instantly.  The obvious advantage of automated processes is speed, but in the case of the airport I think that is pretty much being tossed out, so I see little real reason for it. 

    Datamining is about the same, there will be tons of false positives, and if you don’t have a thinking human being to properly interpret the data, false positives will lead to un-needed arrests, tons of time and money wasted, etc.  Given, it does cut down on time, and using a search query to automatically find piece of data in a database matching it is far faster than a person sitting and sorting it, but again, without proper interpretation, it just causes more problems. 

    Seems to me that increased security measures and tools are just a way to decrease human legwork (and probably eventually cut down on payrolls and such).  Sure they do help increase efficiency and accuracy, but if you reduce or eliminate the human equation, they actually become far less effective (since false arrests are a waste of time and resources).

    One potential but unlikely scenario I can foresee with all the biometric data is that the human side of the equation is either eliminated, reduced to the barest minimum (just the guy that gets the email or whatever from the system to go make the arrest), or the data interpretation is outsourced to save money.  I think the first two possibilities are more likely, I’d hope the decision makers in Washington would realize what a bad idea the latter idea would be, even if it could save lots of money.

  2. Ingolfson, my words exactly. And what Buzz said.

    There is a mentality in the U.S. that you can solve any problem by throwing more technology at it. For many applications, including airline security, I strongly disagree with this stance. The first, second, and third line of defense can only be well-trained staff; scanner technology and the like should be a quality control and last-ditch backup instead of reinforcing a Maginot line mentality.

  3. The problem is two part though.  We want to rely on higher and higher amounts of technology, while at the same time those running things want to cut back on manpower, or just hire those who will do the job for the least amount of money.  That usually means privatizing unionized government jobs, because we all know that unions are the enemy of progress. 

    However for me, I’ll take a union worker over an almost making minimum wage (and has got the minimum training and screening to make sure they are right for the job) private security guard anyday. 

    Unions have their problems sure (I’ve been in 3 that I would say are very bad examples), but for the most part, those in them are “lifers” and have committed to doing their jobs correctly.  Plus there is no replacement for experience in dealing with this sort of thing, and unions encourage this.

  4. Double-dipping.

    In software testing, automation is okay for regression testing. Beyond that, there are often quickly dimishing returns. I’ve done some software testing myself and while my experience is anecdotal, you seem to get the best coverage by tackling the problem from two directions – document the expected behaviour and design a testing plan that covers it, but be sure to hire testers that are smart enough to do all kinds of things that the developers never anticipated. True, with the right kind of skilled nitpickers you’ll never get a product out of the door, but hey…

    Same principle for airline security. If you have the right people and run the operation as a competitive sport(*), I’d feel a lot better about air travel.

    *: Have the security staff of one airport try to breach the security of another – unannounced. Find creative ways to reward success on either side. And so on…

  5. Actually the competive sport idea is a great one. One of the retail stores I worked for in my mispent youth did something similiar (though the odds were stacked in their favor because they pulled 90% of the floor staff away to a meeting while they tried to walk out the store with a mint worth of merch).

    Basically Loss Prevention from the Home Office came in and tested our ability to spot shoplifters and such, by taking what they knew was typical shoplifting tactics.  Like I said, they suceeded because they weighed the odds against us, but the idea has merit.

    I’d like to imagine that already does occur, but considering that appearances are more important than results in most of these cases, I seriously doubt it anything like that occurs regularly.

  6. *: Have the security staff of one airport try to breach the security of another – unannounced. Find creative ways to reward success on either side. And so on…

    I think that’s an excellent idea, elwed. But it is only as good as the staff itself: Let’s say a security person finds a hole in the system—s/he can either report it or expoit it, and it may boil down to whichever is more profitable.

    It still boils down to the personnel.

  7. No one in government at the upper levels would support it though, because it would mean spending money on something that could be tested on a normal, constant, and publically viewable basis. Think Sgt. York, except with people – a sniff of failure would bring out the Blame Police and the Righteous that would continually be scrapping the system. Now, if the airlines did it, and the airlines were given public funds to do it? That might work…But using government employees just seems counter-political.

    That might be good for the public, but the public good isn’t always in step with re-election.

    Besides, with all this technology I’m sure there’s an awful lot of capacity for padding pockets that you just don’t get for providing jobs to people who actually need them.

  8. Elwed: Have the security staff of one airport try to breach the security of another – unannounced. Find creative ways to reward success on either side. And so on…

    The concept is much too intelligent and expensive … and people might enjoy their jobs. smile

  9. El Al does not (only) protect its planes by bomb sniffers and background checks. They talk to you before you board their planes.

    You do not understand their security. El Al has a fraction of the number of airports (Elat, Haifa, Ovda, and Tel Aviv) and flights compared to the US.

    The number of flights in the US x the number of airports makes interviews impractical due to the sheer number of interviews it would require (a 777, for example, seats 275, and about 43 flights—of various capacities—leave LGA every hour, on the order of 384,554 per year).

    How many major airports are there?
    01- Boston (BOS)
    02- Dallas/Fort Worth (DFW)
    03- Detroit (DTW)
    04- Dulles (IAD)
    05- Hobby (HOU)
    06- Houston Intercontinental (IAH)
    07- Kennedy (JFK)
    08- LaGuardia (LGA)
    09- Los Angeles (LAX)
    10- Love Field (DAL)
    12- Midway (MDW)
    13- Newark Liberty (EWR)
    14- O’Hare(ORD)
    15- Reagan (DCA)
    16- San Francisco (SFO)

    We’re talking about literally thousands of flights per day and MILLIONS of flights per year.

    How long would you have to show up for your interview before your flight? 6 months?

  10. The concept is much too intelligent and expensive … and people might enjoy their jobs.

    Guys, its not that this ISN’T done. We keep hearing reports of these teste failing (i.e. the testers getting aboard with dangerous stuff). The fact that we do hear about them should be a positive, but the fact is, these results are not followed up with action and change, because the people/companies actually doing the searching are not rewarded or punished for their behaviour.

    You do not understand their security.

    Thanks for the assumption. While my above description may seem a bit light and airy, I actually know a lot more about it (I just found the sentence to nice an ending for my post to babble on afterwards).

    El Al has a fraction of the number of airports (Elat, Haifa, Ovda, and Tel Aviv) and flights compared to the US.

    You forget that they do the same thing at all other overseas airports they serve.

    Point taken – in fact, EL Al does not even foot the bill. They get many, many millions from the Israeli government for it. Acc. to one report I read, 1/3 of all airport staff in Israel proper are security in some way (not nearly always in uniform though).

    The point is taken and the question raised: do we want to spend the money for our security? And as for the main point of my post: are we spending our money smartly, for systems that fail well?

  11. DDP: You do not understand their security.

    We understand their security perfectly well. Israel and El Al’s top priority is to do make sure no plane of theirs is hijacked or bombed and they are willing to expend the resources to do what it takes.

    The top priority of the U.S. government and airlines is to keep people flying. To add insult to injury, I’m far from convinced that the U.S. gets an acceptable return on investment in terms of improved security. It’s debatable whether the El Al model could possibly scale to U.S. needs, but it’s also debatable if the U.S. could find better uses for the funds spend.

  12. Those tests?

    Ouch.

    Shows what happens if BOTH sides don’t care.

    WHAT GODAMN TESTING AGENCY USES THE SAME TRICKS OVER AND OVER AGAIN?????? HAVE THESE PEOPLE NO GODAMN SPORTING INSTINCT????

    The WORSE thing is that probably the testing agency is (was, I sure hope WAS) forbidden from using other test items, because that would be unfair. Fuck.

    Yeah, there are reasons why your life should not trusted in the hands of a bureaucracy.

  13. Well, it is definitely easier to test a closed system where all the variables are standard and static and known than an open one (oh say like a real live airport…) where things are constantly changing. 

    Given that bureaucracy is all about the numbers, and you can’t really completely quantify an open system, they probably insist on treating it as a closed system so that they can have all the numbers in nice orderly columns to present to the next middle manager up the chain.

  14. so that they can have all the numbers in nice orderly columns to present to the next middle manager up the chain.

    Who pretties it up a bit. Turns it into a powerpoint presentation, maybe. Changes the odd mention of ‘often fails’ into ‘space for improvement’.

    By the time it arrives at anybody really senior (or, mind you, the press), the report is all fluff and kittens.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.