May 12, 2005
Avoiding Liability: An Alternative Route to More Secure Products
Advances in Financial Cryptography (FC++) presents an essay in draft by Adam Shostack on alternates to liability for makers of security software. Can we avoid liability for failed FC apps?
Avoiding Liability: An Alternative Route to More Secure Products
A healthy debate is raging over extending liability rules to software companies. Respected security experts and economists argue that it is an effective way to force companies to internalize externalities. After all, if a company can spend nothing on security, and produce a product that customers will buy, why should they spend on security? If customers can't distinguish between a secure and an insecure product, the company that produces an insecure product will get
to market first, and have an advantage. This shifts the high cost of dealing with insecurities to customers, who are in a poor position to fix the products they have purchased. Thus, imposing liabilities on software producers will induce them all to take care in the creation of their software.
Responses to this argument include that it will dampen entrepreneurship, because a large companies will find it easier to influence, and then comply with the "industry standard" practices that limit their liability. At the same time, corporate executives are focused on trying to limit their liabilities, rather than shift them around (WSJ). This executive opposition, coupled with contract provisions now being imposed by large buyers may be enough to prevent general software liability over the next several years.
What is a customer who wants better software to do? Twenty years ago, there was no good way for a customer to judge the quality of a used car. The dealer knows more about it than the customer reasonably can. It's expensive to bring twenty used cars to your mechanic to get them checked out, and besides, he may see your lemon as his paycheque.
Studying this market earned Akerlof and Spence a Nobel prize: They talk about assymetric information, lemons markets, and signaling, which is a message that's cheap to send if you are a high quality provider, but expensive if you're not.
Today, we have a number of ways of signaling the quality of a used car, including dealer-backed warranties, certified-pre-owned programs, and Carfax, which is a background checking system for cars. Can we take useful lessons from this for security? This essay will first look at some objections to the idea of signaling for security, examine some possible signals, critique those signals, and then compare signals to liability as a means to achieve appropriate security for an organization.
This is our second FC++ presentation, and brings in an important nexus of economic thought to assist us with secure applications. Adam intends to present this essay informally at the rump session of the forthcoming Economics and Security Workshop. Comments by peers can only help! Feel free to distribute the link. Enjoy!
Editor's apology! The email notification to this entry failed consistently 4 times. A mystery.
Posted by iang at May 12, 2005 02:21 AM
Adam concludes that signals about security will be ignored in a new market, so that it makes sense to apply liability to new risky products, but as the novelty matures into a commodity signals work better and liability is a problem as a barrier to entry.
However, liability is also more costly in a new market than a mature one. Given a monopoly, unsophisticated customers, or both, along with unknown risks, vendors want to waive consequential damages, and customers will tend not to challenge this. So contracts will waive liability for bad vendor security. Tort law will be very inefficient because no industry standards or long-standing practice will exists by which to judge what is "reasonable" versus negligent security technology or practice. Nor will the risks, with little or no statistical history, be insurable. Finally, determining the amount of damages incurred from novel attacks may be especially difficult.
Once the product has matured, statistical history allows vendors, customers, and insurers to better estimate and allocate risks using contracts. Tort law works better once industry standards have developed (assuming those standards reflect industry experience and practice rather than a premature codification of wishes, as is all too common with today's Internet-related "standards"). More things have a market value, so damages are easier to assess.
A couple of quick comments from a few pages of reading. I'm searching for my file of other comments :-/
because A large companies ==> because large companies
That section alludes to barriers to entry: "large companies will find it easier to influence, and then comply..." Would be nice if you could just state it as that - a barrier to entry - as this is a well understood term of economics art now (proper reference would be "Porter's five forces").
("`What is security', asked Pilate, and washed his hands.")
Great Quote! I would move it up and blockquote it below the section heading...
"In the market for education, there are perhaps similar difficulties. A student getting ready to chose a college may not have a fair assessment of how they'll fare in the wider world. The valedictorian of a small town high school may have troubles boyond merely adjusting to a top-notch school. If they are choosing a college solely as a signal they could choose either a top flight school, where they may be average or below average, or a second or third tier, where they could excel. (Spence) "
to chose ==> to choose
boyond ==> beyond
Spence directly picked the market for jobs and education as his headline signal in his seminal paper "Job Market Signaling". He also studied sex as a comparison, and his models as presented showed stability in the market using sex (as in male or female) as a signal. (This is not quite the accurate rendition, the paper explains more fully.)
What this means, following Spence, is that a signal is a metric that is used for decision making in investment scenarios that may or may not be useful or correlated. (Spence said specifically that he could not define it.) So, we assume like he does that sex is a bad signal but it is a signal that gets used, and he shows why rationally it reaches stability.
So for Spence, the topic was not the search for "good" or "bad" signals, as he already had a bucket of those. Instead, his paper could be seen as an examination of why certain signals seemed to achieve stability even though they were not useful nor correlated with productive decisions. An inefficiency if you will. (Oddly enough his model is just like Boyd's stuck-in-own-loop syndrome.)
This makes it problematic. It is true that Spence showed that his signal only worked if it was expensive for some to send the signal and cheap for others. But this was not a good thing; as with education, he was not only showing that this was stable without productivity measurement, I think from memory he also showed that *not* using the signal could be more Pareto-efficient.
What does this do to the paper? I'm not sure about this. I think the emphasis is not on saying that signaling is good. Spence was not saying that signals are good. But more that the challenge is to identify what signals were good, and in fact to de-signalise them and turn them into metrics; something that we could rely upon because it helps productivity.
Indeed, for your paper, Spence is suggesting that the signals are already out there but that many of them probably don't relate to the nominal goal of security. It might be for example the name (IBM, etc) or it might be the standard (CC). The danger is that the signal can reach stability but never be causally related to security.
In your above statement, you introduce but don't spell out a dilemma. There are two signals for the valedictorian: a. the name of a school, b. the marks gained at the end. The calculation that needs to be made is whether the importance on the name means going for the top flight school and compromising on grades, or whether the importance of the grades means compromising on the name. I suggest you spell it out, something like "the student faces the challenge of choosing one signal over the other."