Amit Yoran has a certain sort of positive, honest enemy reputation, perhaps reinforced because he actually left the job of DHS cybersecurity head after a year. He was asked by Todd Datz:
What were your biggest frustrations during your tenure at DHS?
"Perhaps a lack of effectiveness in much of the government's security practices, a lack of practicality. There's a phenomenal amount of paperwork around certification and accreditation. There's a significantly sized industry around Washington, D.C., running paperwork exercises on cybersecurity, as opposed to investing in improved operations and implementing security technologies."Take, for example, NIAP [National Information Assurance Partnership]. The Department of Defense says it won't procure any products that haven't been through this certification process. It takes several quarters, if not years, and costs millions of dollars. And what comes out at the end is an approved product for that specific platform, for that version of technology. So you can't apply patches and fixes because it violates your certification. It's a paradigm, an academic exercise. The practical implementation of it-the practical improvement on cybersecurity-is zero. In fact, most people don't even understand what the NIAP certification gets you; it doesn't say your product is secure or doesn't have flaws.
-----
-Todd Datz
Posted by iang at April 5, 2005 07:49 PM | TrackBackSo it's more important to the DoD that their purchased product meet some very specific list, and then *never* get patched? Like that is supposed to be more secure than a fully-patched and up-to-date system?
Sounds much like my unfavorable view of Common Criteria. CC tells you little about how secure a system is or can become, because their evaluation process is probably unlike any that a typical organization would go through. Certification programs are more of a way for purchasers to absolve themselves of responsibility than to do the hard part: evaluate a technology against their own specific requirements.
Posted by: Steve Riley at April 6, 2005 01:31 AMSo I'm afraid I don't know much about the world of security standards, but what are more desirable alternatives? If a policy-maker wanted to specify a level of security (or just an assurance of security) what would be a good mechanism to do this? Given that the systems are going to be procured by contractors looking for minimize costs for their bid, how can the principle extract the most security value from the agent?
Posted by: allan friedman at April 6, 2005 11:51 AMSo here we have a clear contradiction - one group of people think you must apply patches to 'be secure' and another group of people say you mustn't. We need to look fairly closely at what each group is saying ... In fact they are both wrong.
The CC/NIAP people will say simply that if you patch the system, then it is no longer that which got CC'd so we no longer know anything about it, security wise. The patch people will say we have a clear security breach that this patch fixes, if you don't fix it you are clearly insecure.
The difference between the two revolves around their use of the word 'secure'. CC/NIAP people think that if it has passed their standard then it is secure. That's wrong. All we can say is that ... it has passed their standard.
The patch people will say that you have to patch it to make it secure. No, as even if patched, it won't be secure. That just means that the hole is patched, maybe.
What is security? It is not an absolute, and there is no action nor standard nor best practice that can make it so. Security is a relative concept; and a complex one at that. Which means that standards won't make us secure, and a parade of patches won't keep us secure. Rather, what we need is risks & returns analysis: good old fashioned engineers who understand what the meaning of these standards and patches is and whether to apply them in each case.
Posted by: Iang at April 6, 2005 12:22 PM