I recently received an (anonymous) comment on the 'silver bullets' paper that ran like this:
Sellers most certainly still have more information than the vast majority of buyers based on the fact that they spend all of their time making security software.
That's an important statement, and deserves to be addressed. How can we check that statement? Well, one way is that we could walk over to the world's biggest concentration of sellers and perhaps buyers, and test the waters? The RSA conference! Figuratively, blog-wise, Gunnar does just that:
I went to RSA to speak with Brian Chess on Breaking Web Services. First time for me to RSA, I generally go to more geek-to-geek conferences like OWASP. It is a little weird to be in such a big convention. There were soooo many vendors yet most of the products in the massive trade show floor would have as much an impact on the security in your system as say plumbing fixtures. What is genuinely strange to me is that every other area in computers improves and yet security stagnates. For years the excuse that security people gave for their field's propensity to lameness is that "no one invests a nickel in security." However, that ain't the case any more and yet most of the products teh suck. This doesn't happen in other areas of computing - databases are vastly better than a decade ago, app servers same, OS same, go right down the list. What gives in security? Where is the innovation?
This is more or less similar to the paper's selection of quotes. Anecdotally, evidence exists that insiders don't think sellers know enough, on both sides of the fence. However, surveys can be self-selecting (as was my sample of quotes in the paper), and opinions can be wrong. So it is important to realise that we have not proven one way or another, we've simply opened the door to an uncertainty.
That is, it could be true that sellers don't know enough! How we then go on to show this, one way or another, is a subject for other (many) posts and possibly much more academic research. I don't for a moment think it is reasonable nor scientifically appropriate to prove this in one paper.
Posted by iang at April 16, 2008 07:01 AM | TrackBackSpeaking as someone who used to work for computer software companies, I find it perfectly plausible that software vendors could know less than their customers, or at least less than some large fraction of their customers. A lot of the things traditional software companies base product plans on (internal engineering ideas, analyst advice, competitive matrices, etc.) don't necessarily reflect real knowledge about what works and what doesn't. The problem is exacerbated in the security field because of the dearth of real-life information needed to create such knowledge.
Posted by: Frank Hecker at April 16, 2008 11:14 AM