September 20, 2008
Builders v. Breakers
Gunnar lauds a post on why there are few architects in the security world:
Superb post by Mark on what I think is the biggest problem we have in security. One thing you learn in consulting is that no matter what anyone tells you when you start a project about what problem you are trying to solve, it is always a people problem. The single biggest problem in security is too many breakers not enough builders. Please understand I am not saying that breakers are not useful, we need them, and we need them to continue to get better so we can build more resilient systems. But the industry is about 90% breaking and 10% building and that's plain bad.
It’s still predominantly made up of an army of skilled hackers focused on better ways to break systems apart and find new ways to exploit vulnerabilities than “security architects” who are designing secure components, protocols and ultimately secure systems.
Hear hear! And why is this? One easy answer: breaking something is a solid metric. It's either broken or not, in general. Any journo can understand it.
On the other hand, building it is too difficult a signal. There is no easy number, there is no binary result. It takes a business focus over decades to understand that one architecture delivers more profits for users and corporates alike than another, and by then, the architects have moved on, so even, then the result may not be clear.
Let's take an old example. Back around 1996, a couple of bored uni students cracked Netscape's secure browsing. The first time was by crunching the 40 bit crypto using the idle lab computers, and the second time was by predicting the less-than-random numbers injected into the protocol. These students were lauded in the press for having done something grand.
They then went on to a much harder task, and were almost never heard of again. What was that harder task? Building secure and usable systems. One of them tried to build a secure communications platform, another is trying to build a secure computing platform. So far, they have not succeeded at either task, but these are much harder tasks.
The true heroes back in the mid-1990s were the Netscape engineers who got something going and delivered it to the public, not the kids who scratched the paint off it. The breaches mentioned above were jokes, and bad ones at that, because they distracted attention on what was really being built. Case in point, is that, even today, if we had twice as much 40 bit crypto as we do 128 bit crypto, we'd probably be twice as secure, because the attack of relevance simply isn't the bored uni student, it is the patient phisher.
If you recall names in this story, recall them for what they tried to build, and not for what they broke.
Posted by iang at September 20, 2008 05:59 AM
we had been brought in to consult with this small client/server startup that wanted to do payment transactions on their server ... and they had this technology they had invented called SSL which they wanted to use. in previous life, we had worked with the two people responsible for the "commerce" server (on scaleup for high integrity, high availability, high thruput DBMS).
we had sign-off on various aspects of the server to payment gateway operation ... so could mandate various things about industrial strength networking (before things like internet SLA contracts, so had to invent/design compensating processes). lots of past posts
however we weren't allowed to make corresponding mandates about the interface between client and the server. we did some tutorials about industrial strength networking. For instance, one of the things we mandated for the server/gateway interface was multiple-A record support (with the payment gateway having multiple connections to different parts of internet backbone)... for availability.
We had spec'ed multiple interfaces for some number of deployed commercial "commerce" servers (for their availability) and "suggested" that client needed multiple-A record support. Initial response was that multiple-A record support was "too advanced" ... even after we provided client multiple-A record support from 1988 4.3 tahoe tape. It took another yr before there was multiple-A record support in the client.
I liked this one. I'm a builder. Actually I think I *ought* to get more practice at breaking things, because as Ferguson and Schneier have always said, this is necessary experience for good building. Well, hopefully they weren't quite right.
Did I tell you about the Hack Tahoe! contest? :-)
See also my recently completed academic publication:
I think you've oversimplified things a bit. A common lament, repeated in this blog post, is that it's difficult to distinguish secure systems from insecure systems. That ability is essential to the creation of a market for secure systems. Breakers provide a signal (a noisy one, to be sure) that can be useful for distinguishing secure systems from insecure systems.
Consider the recent work on analysis of e-voting systems, for instance. I would argue that the work of Breakers has had an enormous positive impact, and has been essential for progress in that field. Without the demonstration of vulnerabilities, no one was willing to pay for improving the security of these systems -- and Builders are irrelevant if no one wants to hire top-notch Builders and give them the resources they need to do an excellent job.
http://www.garlic.com/~lynn/2008n.html#34 Builders v. Breakers
work on earlier high-integrity, high-availability networking (and scalable cluster DBMS) ... we had done a detailed vulnerability analysis of network specifications and code implementations. Most common security issues were viewed as just part of deploying industrial strength operation.
For the server/gateway interface we had developed a failure matrix for different states ... and required that all possible cases were covered. We've frequently commented in the past that taking a well tested/designed application and creating industry strength service ... requires 4-10 times the original effort.
In the time frame of the payment gateway ... there was a situation where the largest online service provider was experiencing failures on their internet interface. After two months of having numerous experts come in and look at the problem, one of the people flew out to the west coast and bought me a hamburger after work. While I ate the hamburger, they explained the symptoms. I then said that was one of the top ten problems we had earlier identified in detailed vulnerability analysis and gave them a Q&D fix which they installed later that night.
'None' brings up an interesting point; the need for, or useful service of breakers to distinguish between secure and insecure systems.
Sure, in a static sense. Unfortunately, there is both dynamics and an interest incentives. The builders are brought in early and are generally disposed of well before any large volumes are seen, as by definition, it's working well before then, so we need no more building upwards, just outwards. In contrast, the breakers only turn up when the large volumes have exceeded a certain _growth point_. This will be later on.
So, even if the signal is distinguishable, it is generally too late, by itself. Indeed, it leads to poor interpretation of the signal, in that a manager might take that ability to break as a signal that the breaker can be a better builder. And so the downwards spiral begins...
http://www.garlic.com/~lynn/2008n.html#34 Builders v. Breackers
http://www.garlic.com/~lynn/2008n.html#35 Builders v. Breackers
along the line of industrial strength dataprocessing there are periodic articles about the general poor quality of many software products and the large number of defects (not limited to security defects).
separate from the perceived peer "points" from breaks, there are also the claims that products are generally being shipped and letting users/customers "debug" the product (development skimping on time, skills, money).
a couple past posts about holding a mini-conference at our house a decade ago for cal. computer security graduate program and discussing the difficulty of getting students interested in building rather than breacking:
http://www.garlic.com/~lynn/2005c.html#26 [Lit.] Buffer overruns
http://www.garlic.com/~lynn/2007u.html#62 folklore indeed
Iang laments that Breakers show up too late to be useful. That's not universally true. If you pay for Breakers to analyze your system before the system is widely deployed, then you can certainly get the signal early in the lifetime of the system. On the other hand, if you aren't willing to pay, what do you expect?
If you expect people to provide Breaking for free, then you shouldn't be surprised if it comes at a later point in the lifecycle of the system.
Even when the Breaks comes later in the lifecycle of a system, they can still be useful to some buyers. And they might still be useful at providing vendors a partial incentive to improve security (because they know that if they build a lousy system, it may get broken after it is deployed but before they've fully reaped all the profits they might hope for). It seems plausible that the knowledge that Breakers exist out there will help to drive decisionmakers to spend more on Builders.
Moreover, this discussion has not considered the benefits to the field of breaks on systems. In some cases, those breaks can help us to understand better how systems fail, which can help us to make future systems more secure. There's a reason why the top research conferences publish break papers.
There's also a reason why AES is so good, and it has partly to do with the amount of effort put into cryptanalysis over the past decades.
Finally, I draw attention again to my example of e-voting.
I don't think this is as black-and-white as it's been made out to be, and I don't think it's healthy to try to paint Breakers = bad, Builders = good. It's not that simple.
None -- There's little that is black-and-white here, but we do know that the classical security field has failed to deliver. So the search is on for the reasons why, and for alternates.
Yes, in that search we are going to sound necessarily harsh to a group of people found on the wrong side of the fence. It may be that their service is not valueless. It is possible to deliver local value while being part of the problem and not the solution. Someone has to do the dirty work, and collecting garbage can be well paid. (Why paying for the breakers earlier is not going to work is discussed in depth in the GP rants.)
It is not that the Breakers are "the problem," more that they are "more part of the problem side than of the solution side." The emphasis on breakers is symptomatic of the emphasis on bolt-on security, as driven by media attention. Yes, if the voting world is truly broken, truly a mess, it may be that the only way forward is via the breakers & media.
But it isn't a thoughtful, chosen, viable or preferred strategy. To positively laud the breakers is to miss the wider systemic issues of we apparently need them rather than doing it properly up-front.
E.g., a similar mistake is to draw lessons from crypto into wider security (and indeed zooko points out that the breakers are lauded in the crypto world!). Crypto is such a narrow and specialised field, that practically everything it says --while relevant to its own small island-- is useless in the wider world. Whining that "if only it were more like AES" is not going to help, and it will probably harm. Only crypto is like crypto, the rest of the world is squishy, soft and not amenable to crypto-like recommendations.
E.g., SSL came out of the crypto field. I frequently recommend more and more SSL everywhere as a way "forward". (Have they implemented TLS/SNI yet? No! Why not, we could validly ask? Do you know why they haven't implemented TLS/SNI yet?)
But this hides a more vital truth, that SSL-as-security is like protecting virginity with knickers. Both are point-to-point, lower layer plugins, so they cannot possibly deliver end-to-end, application-purposed security. By definition! Anyone who believes these things will also be trapped in security poverty for generations.
But, for the HTML world that is stuck there, currently believing that SSL is their security, it still remains good advice to them. "Don't you dare go out there without your SSL on!" (See the naked-and-vulnerable thread for more...)