Financial Cryptography

Security Signalling - the market for Lemmings

Adam continues to grind away at his problem: how to signal good security. It's a good question, as we know that the market for security is highly inefficient, some would say dysfunctional. E.g., we perceive that many security products are good but ignored, and others are bad but extraordinarily popular, and despite repeated evidence of breaches, en masse, users flock to it with lemming-like behaviour.

I think a real part of this is that the underlying question of just what security really is remains unstudied. So, what is security? Or, in more formal economics terms, what is the product that is sold in the market for security?

This is not such an easy question from an economist's point of view. It's a bit like the market for lemons, which was thought to be just anomalous and weird until some bright economist sat down and studied it. AFAIK, nobody's studied the market for security, although I admit to only having asked one economist, and his answer was "there's no definition for *that* product that I know of!"

Let's give it a go. Here's the basic issue: security as a product lacks good testability. That is, when you purchase your standard security product, there is no easy way to show that it achieves its core goal, which is to secure you against the threat.

Well, actually, that's not quite correct; there are obviously two sorts of security products, those that are testable and those that are not. Consider a gate that is meant to guard against dogs. You can install this in a fence, then watch the rabid canines try and beat against the gate. With a certain amount of confidence you can determine that the gate is secure against dogs.

But, now consider a burglar alarm. You can also install it with about the same degree of effort. You can conduct the basic workability tests, same as a gate. One opens and goes click on closing; the other sets and resets, with beeping.

But there the comparison gets into trouble, as once you've shown the burglar alarm to work, you still have no real way of determining that it achieves its goal. How do you know it stops burglars?

The threat that is being addressed cannot be easily simulated. Yes, you can pretend to be a burglar, but non-burglars are pretty poor at that. Whereas one doesn't need to be a dog to pretend to be a dog, and do so well enough to test a gate.

What then is one supposed to do? Hire a burglar? Well, let's try that: put an ad in the paper, or more digitally, hang around IRC and learn some NuWordz. And your test burglar gets in and ... does what? If he's a real burglar, he might tell you or he might just take the stuff. Or, both, it's not unreasonable to imagine a real burglar telling you *and* coming back a month later...

Or he fails to get in. What does that tell you? Only that *that* burglar can't get in! Or that he's lying.

Let's summarise. We have these characteristics in the market for security:

• a test of the product by a simulated threat is:
  • expensive, and
  • the results cannot be relied upon to predict defence against a real threat;
• a test of the product by a real threat is:
  • difficult to arrange,
  • could be destructive, and
  • the results cannot be relied upon to predict defence against any _other_ real threat.

Perhaps some examples might help. Consider a security product such as Microsoft Windows Operating System. Clearly they write it as well as they can, and then test it as much as they can afford. Yet, it always ships with bugs in it, and in time those bugs are exploited. So their testing - their simulated threats - is unsatisfactory. And their ability to arrange testing by real threats is limited by the inefficient market for blackhats (another topic in itself, but one beyond today's scope).

Closer to (my) home, let's look at crypto protocols as a security product. We can see that it is fairly close as well: The simulated threat is the review by analysts, the open source cryptologists and cryptoplumbers that pore through the code and specs looking for weaknesses. Yet, it's expensive to purchase review of crypto, which is why so many people go open source and hope that someone finds it interesting enough. And, even when you can attract someone to review your code, it is never ever a complete review. It's just what they had time for; no amount of money buys a complete review of everything that is possible.

And, if we were to have any luck in finding a real attacker, then it would only be by deploying the protocol in vast numbers of implementations or in a few implementations of such value that it would be worth his time to try and attack it. So, after crossing that barrier, we are probably rather ill-suited to watching for his arrival as a threat, simply due to the time and effort already undertaken to get that far. (E.g., the protocol designers are long since transferred to other duties.) And almost by default, the energy spent in cracking our protocol is an investment that can only be recouped by aggressive acquisition of assets on the breach.

(Protocol design has always been known to have highly asymmetric characteristics in security. It is for this reason that the last few years have shown a big interest in provability of security statements. But this is a relatively young art; if it is anything like the provability of coding that I did at University it can be summarised as "showing great potential" for many decades to come.)

Having established these characteristics, a whole bunch of questions are raised. What then can we predict about the market for Lemmings? (Or is it the market for Pied Pipers?) If we cannot determine its efficacy as a product, why is it that we continue to buy? What is it that we can do to make this market respond more ... responsibly? And finally, we might actually get a chance to address Adam's original question, to whit, how do we go about signalling security, anyway?

Lucky we have a year ahead of us to muse on these issues.

Posted by iang at January 2, 2005 12:24 AM | TrackBack