March 17, 2012

Fans of Threat Modelling reach for their guns ... but can they afford the bullets?

Over on New School, my threat-modelling-is-for-birds rant last month went down like a lead balloon.

Now, I'm gonna take the rap for this one. I was so happy to have finally found the nexus between threat modelling and security failure that has been bugging me for a decade, that I thought everyone else would get it in a blog post. No such luck, schmuck. Have another go!

Closest pin to the donkey tail went to David, who said:

Threat modeling is yet another input into an over all risk analysis.

Right. And that's the point. Threat modelling by itself is incomplete. What's the missing bit? The business. Look at this gfx, risk'd off some site. This is the emerging 31000 risk management typology (?) in slightly cut down form.

The business is the 'context' as shown by the red arrow. When you get into "the biz" you discover it's a place of its own, a life, a space, an entire world. More pointedly, the biz provides you with (a) the requirements and (b) a list of validated threats. E.g., history, as we already deal with them.

The biz provides the foundation and context for all we do - we protect the business, without which we have no business meddling.

(Modelling distraction: As with the graphic, the source of requirements is often painted at the top of the diagram, and requirements-driven architecture is typically called top-down. Alternatively and depending on your contextual model, we can draw it as a structural model: our art or science can sit on top of the business. We are not lifting ourselves up by our bootstraps; we are lifting the business to greater capabilities and margins. So it may be rational and accurate to call the business a bottom-up input.)

Either way, business is a mess, and one we can't avoid. We have to dive in and absorb, and in our art we filter out the essence of the problem from the business into a language we call 'requirements'.

Then, the "old model" of threat modelling is somewhere in that middle box. For sake of this post, turn the word risk into threat. Follow the blue arrow, it's somewhere in there.

The point then of threat modelling is that it is precisely the opposite of expected: it's perfect. In more particular words, it lives without a context. Threat modelling proceeds happily without requirements that are grounded in a business problem or a customer base, without a connection to the real world.

Threat modelling is perfect by definition, which we achieve by cutting the scope off at the knees.

Bring in the business and we get real messy. Human, tragic. And out of the primordial swamp of our neighbors crawls the living, breathing, propogating requirements that make a real demand on us - survival, economy, sex, war, whatever it is that real people ask us for.

Adam talks about Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service and Elevation of Privilege... which sounds perfect. And I'd love to play the game :)

But real life is these things: insider trading, teenager versus teenager versus parent versus parent, revolutions of colour, DVD sharing, hiding in plain sight, movement tracking of jealous loved ones, don't ask, don't tell, whistleblowing, revenge,...

Which philosophy of business context first and foremost also explains a lot of other things. Just by way of one example, it gives an essential clue as to why only end-to-end security is worth anything. Application security automatically has a better chance of including the business; point-to-point designs like IPSec, SSL, DNSSec, etc have no chance. They've already handwaved anything past the relevant nodes into utter obscurity.

Posted by iang at March 17, 2012 05:54 PM | TrackBack

to throw a monkey wrench into the whole thing ... nominally institutions are doing threat modeling as part of evaluating their risks, leading to (self-interest) taking countermeasures to the risks (a variation on "security proportional to risk" ... specifically the risks to the institution).

this is a discussion in a linkedin mainframe group about sensitive data
and my contributions archived here

the issue is that there is large class of threats involving copying sensitive data ... but where things get short circuited ... the risk is not to the institution ... but to individuals. aka the institutions have no risk. The challenge then was trying to come up with ways to motivate institutions to take countermeasures (where the institutions have no risk and self-interest).

The specific instances was HIPPA (to motivate institutions to protect individual PII information) and data breach notification legislation (possibly indirectly motivating institutions to protect individual financial transaction information).

For other side-track, SSL has been used to hide financial transaction information ... for point-to-point transaction transmission. However, that is trivial portion of the end-to-end transaction business process (especially taken from the standpoint of the elapsed time of the transmission compared to the total lifetime that the transaction information continues to a represent a risk).

As periodically mentioned before, we were involved in deployment of SSL for doing payment transactions on webservers ... now frequently called "electronic commerce" ... nearly 20yrs ago now ... and for almost the whole time I've pointed out that SSL is only trivial part of the end-to-end business process.

Posted by: Lynn Wheeler at March 17, 2012 10:06 AM

So...shoot me your mailing address already. :)

Posted by: Adam at March 17, 2012 10:59 AM

Oh, and I think a big chunk of the problem is that threat modeling is an aspirational kitchen sink. Everyone projects their dreams and aspirations into the very big bucket. You can't say "threat modeling is..." and get away with it (ahem) because it means too many things to too many people.

That's why I tend to say things like "STRIDE per Element Threat Modeling" or "threat modeling as done at Microsoft in the early 2010s"

Posted by: Adam at March 17, 2012 11:02 AM


Hmm, first an updated old joke,

A young couple are honeymooning by driving around the country side in their much cherished vintage touring car. It's been a nice few days but now they have a little problem, they are lost in the scenery and don't know how to get where they want to go.

Just then the groom spots a farmer leaning on a gate and decides to stop and ask directions.

He asks the farmer and is surprised by what happens. The farmer listen attentivly looks very thoughtfully over to the old banger with the bride sitting in it, knocks his pipe out and relights it before coughing to clear his throat and say "Well two things first your old banger isn't green and secondly if I was you I wouldn't start from here".

Such is the state of security the landscape is large and it's easy to get lost in, which is why many people need clear and simple directions, they don't particularly care if it's the best, nicest, fastest or most economical route all they care about is getting to the next destination on their journy.

What they don't realy have time to be told is there are better modes of transport they just want to get somewhere so they can move on again.

So any instructions they get are only going to be useful if and only if they understand them and they ar thus simple to follow and importantly relevent to the way they already do things.

Posted by: Clive Robinson at March 17, 2012 03:20 PM

I somewhat agree that threat-modelling is an input to risk-analysis.
Well, in one project, I started with threat-modelling, but then I didnīt got to the next point of doing a risk-analysis. What I liked about the threat-modelling is that due to forcing myself to think it through, I both discovered a few new threats, and I developed a threat-meta-model that helped to understand the whole business in a new way.
In another project, I had to risk-analysis nearly every day in a rather ad-hoc manner, but due to the large continuously changing business, it did not seem easy to do a formal risk-analysis.
So doing threat-modelling had positive results (better understanding of the business and finding new threats) for the time and energy it took. With risk-analysis, I guess I have not found a way yet that works for me, or perhaps I am doing things in a completely wrong way.
I agree that one important aspect is compartmentalisation. Compartmentalisation is what makes threat-modelling and risk-analysis difficult, but threat-modelling and risk-analysis help to fight the downsides of compartmentalisation. (Why is there mentalisation in the word compartmentalisation?!?)
One thing that threat-modelling can help is to collect and document threats over a longer period of time to advise newcomers about problems that happened before their time in the business. Perhaps we we should call it threat-collection or threat-historicanism or something, to show that it should not be just a single step during the development/creation, but that it should also be updated somewhat regularly during the lifetime of the business.
One thing that I like about threat-models is that they are not as personal (/confidential) as risk-analysis(es?). In a risk-analysis, you have to tie the threats to your business, and e.g. state, whether mechanism X (which you already deployed) already prevents threat Y, so itīs not a risk anymore, ...
So if you disclose your risk-analysis, you also disclose, that you deployed mechanism X.
But disclosing your threat-model is not so sensitive, so it can help the security community to learn from it, if you publish it. (Threat-catalogues ...)

Posted by: PG at March 17, 2012 05:13 PM

and in the theme ... of "can they afford the bullets" ... another facet of (often repeated over the years) "Security Proportional To Risk" ... with respect to financial transactions

the value of the financial transaction information to the merchant is the profit on the transactions ... possibly a couple dollars (possibly only a few cents to the transaction processor). The value of the financial transaction to the crooks is the account balance and/or credit limit ... typically hundreds to thousands of dollars. As a result, the crooks can potentially afford to out-spend by 100 times, or more, attacking the system ... as the merchants/transaction-processors can afford to spend defending the system.

this is separate from the fact that exploit risk is to the individual ... not the merchant or transaction processor.

Posted by: Lynn Wheeler at March 18, 2012 10:35 AM

yes, precisely. I think I have to disclose that really, I'm taking aim at those mid-late 1990s security practitioners who wielded the "what's your threat model?" weapon to great effect. Without that threat modelling exercise being linked to the business or user requirements, it was far too easy to use it as a blunt weapon to ensure the security model already chosen was enforced. In other words, the modelling was to ensure the chosen result.

Still, we do know at some level that threat modelling has to link to the business requirements. Problem is, why doesn't this work? I think, on reflection, the biggest problem is that the people who are very good at security aren't as good at business. Unfortunately if someone is going to spend the time and money to do such a job properly, that's not a good enough excuse.

But it would be interesting to hear whether you saw much the same thing in Microsoft. I'm guessing maybe only flavours of it.

Posted by: Iang at March 18, 2012 07:23 PM


yes, that is a fair observation. But it does mean that the landscape can't really be appreciated in any efficient fashion by travellers. It also puts the traveller at the mercy of strangers, well-meaning farmers and outright shysters. Maybe we should just admit it - security is the wild west, and this is why waggon trains like firewalls & SSL were such a good idea?

I think this is why I like the advice of doing the entire security work oneself, even if one isn't trained in it. At least the business will be first and foremost in ones thoughts. And, even if you do a worse job than some practitioner promises he could achieve, at least you will have earnt the experience to be able to fix it next time. Something you won't be able to do if the story is reversed.

Posted by: Iang (H6) at March 18, 2012 07:32 PM

I have a similar jaundiced view of a lot of the threat model stuff in the mid-late 90s ... Lots of parties had the solution (PKI) and they were using the facade of threat models to narrowly focus in portion of problem needing PKI as solution (they had the answer, now they needed to find the question). The industry was floating business plans on wall street $20B/annum for annual $100 digital certificates for individuals. We had been called in to help wordsmith the cal. state electronic signature legislation ... and the industry was heavily lobbying that the legislation mandate (annual, individual) digital certificates.

Posted by: Lynn Wheeler at March 19, 2012 10:49 AM
Post a comment

Remember personal info?

Hit preview to see your comment as it would be displayed.