January 17, 2009
Getting the business into security, or is it...
Ian says in comments to the post on "Business":
Your emphasis - exactly. I read Frank's 'paper' yesterday and I read it very differently. You've missed emphasising "security is essentially risk management" in the first sentence. i.e. Frank IS saying that economic risk is the turning point of the whole thing.
yes, clearly risk management is how they link their security model approach to the business. My point however was that this was a "nod" and not necessarily enough.
Let's make this polemic. Risk management is a dead duck. Here's some reasons why:
The only business that does risk management as a core or essence is banking and insurance (and, banking is debatable these days). For all others, risk management is just an ancilliary aspect, a nice-to-have, something that others say is critical to you, but you ignore because you've got too much to do. So we have a choice: is security like finance, or is it like "the rest of business?"
I would say it is not like finance. So risk management is not the core.
The question then might be whether risk management as an ancillary adds anything that helps? That depends, a lot. It turns out there is a fatal flaw in this approach.
What is the risk management approach? Well, at the detailed level, it generally turns out to be something like two calculations:
risk = (percentage chance of event) * (damage/costs of when it happens.)
defence = (percentage chance of mitigation) * (money saved)
result = comparison_function (set of all risks, set of all defences, costs).
We really don't need to cite a lot of papers (security academics take note) nor get hung up on what the real meaning of the words or variables are here, because this is a well known finance technique. It's called ROI, or more properly NPV. Let's just borrow from the finance people, because they have done this work, won their Nobel prizes and covered the territory.
Frequently, it is pointed out that the financing of security projects should be done on this basis. This is true because we don't have any other cross-business comparison tools, and your CFO demands it.
However, regardless of this truth, it doesn't really satisfy with security projects. The reason NPV doesn't work is that we don't have good numbers to plug in, like those that we have in finance. ROI in infosec is GIGO, whereas for other business areas, all of them, we can actually find those numbers. (There are good reasons why this is the case, and the hint here may be that security is like defence, and they don't do good ROI either.)
So, NPV doesn't work in Security, even though we need it. Risk management is just another word for NPV, so risk management doesn't work. Although the theory is pretty cool, actually, we don't know what those numbers are (a priori, risk management suffers GIGO), and afterwards, as long as we are making profits, we don't care (a posteriori, profits are more important than risks).
What's left? In both cases, the discussion is swamped by business issues, and those issues don't give a hoot for either number. What's left is business. If we haven't seen security as a business problem, first and foremost, no amount of Markovitzian mathematics is going to save us.
Consider the famous case of the car-lock. Car locks used to be notoriously weak. Why? Because a car stolen was a car sold. So, no matter what numbers were applied in the above risk management calculation, it always gave the wrong result; better locks made the position worse!
The simple view of this is "What's your business model?" If you want to put it in a more academic strain of thought, then yes, it is economics, but we have to include liability dumping as a technique, and that is not something that is mathematically pliable. Better to skip the econ approach, and just call it for what it is: business.
Posted by iang at January 17, 2009 03:48 PM
I find that most people with InfoSec backgrounds confuse the purpose of using probability theory in risk analysis (1).
Most security folks (and many in the financial industry) believe that risk analysis is something to *engineer* future state, rather than a tool used in understanding our ability to meet qualitative objectives. As such, when the state of nature changes (as it inevitably does) or when it's determined that the analyst screwed up in accounting for uncertainty or variable measurement - the whole process is demonized.
In reality, a good model for risk analysis can only help rational actors arrive at rational conclusions. It cannot and will not forsee a precise future state, but it rather serves to help remove bias and provide structure to what would otherwise be an ad-hoc decision making process. It is with this in mind, that I often ask the authors of these sorts of articles - "well, how then shall we live?" The best answer I get is "suggested practices"(2). The problem with this concept is that it is, in and of itself, a risk analysis model, just one done as a faith-based initiative rather than one done with any real rigor ("trust me, I'm the auditor, you need these controls").
W/regards to other points:
"The only business that does risk management as a core or essence is banking and insurance"
False on two accounts. First, allow me to point you to future earnings guidance statements made by public companies.
Second, I'd say that FinServ is just a market segment that applies analytical rigor to a product line that has a significant degree of uncertainty. Chemical and Aerospace engineering, Food Service, and many other industries I'm skipping over do perform rigorous risk analysis, it's just that the system they operate in has much less uncertainty.
"risk management is... something...you ignore because you've got too much to do."
Nope, at worst it's just something you don't apply significant rigor to because it's not perceived as necessary. When you walk across the street, decide to hire or not to hire, just about any decision that has the potential for negative consequence, you're creating a belief statement that is "go" or "no go". This is very much a risk analysis, as in a Bayesian sense you're creating a belief statement about what is the most probable wise action.
"ROI in infosec is GIGO"
I think you're confusing the concept of the quality of inputs into a model with a statement about the quality of the model.
With regards to ROI in infosec, I find those who simply state that it "can't be done" categorically to be boorish purveyors of hyperbole. They seem to be obsessed with confidentiality and forget that availability is a significant aspect of the charter for most security departments. ROI for keeping production systems available most certainly can be calculated with some degree of suitability.
Now that said, I don't believe that ROI is applicable when we're concerned with and/or including the probability of losses due to breaches in confidentiality and integrity, as these concepts are not easily tied to incoming cash flow in a direct and obvious manner.
"Risk management is just another word for NPV, so risk management doesn't work."
False premise, false conclusion. NPV necessitates some concept of cash flow: Rt/(1+i)^t where Rt is cash flow in. Risk Analysis, in InfoSec/Engineering at least, is currently based on the Dutch model: probable frequency of loss and probable magnitude of loss (note that ALE is a number of limited value, as risk is a derived number like km/hr). Two totally different concepts.
"a priori, risk management suffers GIGO"
Um, what? If you mean that using deductive reasoning, models about the world require useful inputs to develop useful outputs, OK then. All perceptions of reality have that same limitation. But I see no deduction on your part to achieve a statement of "a priori".
"Consider the famous case of the car-lock. Car locks used to be notoriously weak. Why? Because a car stolen was a car sold. So, no matter what numbers were applied in the above risk management calculation, it always gave the wrong result; better locks made the position worse!"
You seem to be assuming an objective ethical position here and inferring that all actors would desire to achieve it. Rather, the car company most certainly did an analysis and came to the conclusion that it's interests were different than the consumer. It's a great example not because it "proves" risk analysis to be silly in some Popperist sense (3) but rather it highlights the most interesting problem in Risk Management - the problem of multiple perspectives (an example would be where the risk manager's individual compensation is inconsistent with executive risk tolerance).
Finally, in response to your summary, I think you over-complicate the value the CISO/CSO/CRO has to the company. Their value boils down to only two things; Align risk exposure to the tolerance of management or create operational efficiencies. All this other talk of "aligning to business and strategy" is, in my opinion, pure bunk.
(1): note that the concept of risk management isn't necessarily what you're referring to here - risk management has as much to do with understanding capability as it does with arriving at a state of knowledge. Without that capability component, you'll never achieve a state of wisdom.
(2): ironically using the term "best/good practices" implies some sort of analysis and measurement.
(3): In fact, I'd say that the state has changed to the point where the opposite is true, cars probably have too much lock security built in. I wonder what the locksmithing industry would have to say about the 70's vs. now and their ability to retrieve our keys for us.