October 26, 2011

Phishing doesn't really happen? It's too small to measure?

Two Microsoft researchers have published a paper pouring scorn on claims cyber crime causes massive losses in America. They say itís just too rare for anyone to be able to calculate such a figure.

Dinei Florencio and Cormac Herley argue that samples used in the alarming research we get to hear about tend to contain a few victims who say they lost a lot of money. The researchers then extrapolate that to the rest of the population, which gives a big total loss estimate Ė in one case of a trillion dollars per year.

But if these victims are unrepresentative of the population, or exaggerate their losses, they can really skew the results. Florencio and Herley point out that one person or company claiming a $50,000 loss in a sample of 1,000 would, when extrapolated, produce a $10 billion loss for America as a whole. So if that loss is not representative of the pattern across the whole country, your total could be $10 billion too high.

Having read the paper, the above is about right. And sufficient description, as the paper goes on for pages and pages making the same point.

Now, I've also been skeptical of the phishing surveys. So, for a long time, I've just stuck to the number of "about a billion a year." And waited for someone to challenge me on it :) Most of the surveys seemed to head in that direction, and what we would hope for would be more useful numbers.

So far, Florencio and Herley aren't providing those numbers. The closest I've seen is the FBI-sponsored report that derives from reported fraud rather than surveys. Which seems to plumb in the direction of 10 billion a year for all identity-related consumer frauds, and a sort handwavy claim that there is a ration of 10:1 between all fraud and Internet related fraud.

I wouldn't be surprised if the number was really 100 million. But that's still a big number. It's still bigger than income of Mozilla, which is the 2nd browser by numbers. It's still bigger than the budget of the Anti-phishing Working Group, an industry-sponsored private thinktank. And CABForum, another industry-only group.

So who benefits from inflated figures? The media, because of the scare stories, and the public and private security organisations and businesses who provide cyber security. The above parliamentary report indicated that in 2009 Australian businesses spent between $1.37 and $1.95 billion in computer security measures. So on the reportís figures, cyber crime produces far more income for those fighting it than those committing it.

Good question from the SMH. The answer is that it isn't in any player's interest to provide better figures. If so (and we can see support from the Silver Bullets structure) what is Florencio and Herley's intent in popping the balloon? They may be academically correct in trying to deflate the security market's obsession with measurable numbers, but without some harder numbers of their own, one wonders what's the point?

What is the real number? Florencio and Herley leave us dangling at that point. Are they are setting up to provide those figures one day? Without that forthcoming, I fear the paper is destined to be just more media fodder as shown in its salacious title. Iow, pointless.

Hopefully numbers are coming. In an industry steeped in Numerology and Silver Bullets, facts and hard numbers are important. Until then, your rough number is as good as mine -- a billion.

Posted by iang at October 26, 2011 05:05 PM | TrackBack


The two Microsoft researchers benifit by getting their names in the press again as does Microsoft.

Arguably as Microsoft can be said to be complicit in the losses (ie it's their OS/Apps/development tools/etc/etc underlying the exploit) it's definitely in their interest to minimise the losses.

However these losses are not realy accuratly reportable in the first place, because we don't know what to measure or even how, they are at best inflated "best guesses", at worst a deliberate attempt to deceive for gain either directly or indirectly (ie they are fradulant).

But even when not guessing or commiting fraud the whole process is worthless because it's a little like saying,

We get around X thousand spam messages a year and it takes on average Y seconds for a recipient to delete one and our average employee cost per hour is Z...

The first question is where do the XY&Z averages come from and what is the spread. But it misses the point if the lowest paid workers delete spam faster than the higest paid (which is quite likely) then the figures are going to be out by way more than a "country mile".

Then if you put inplace technical measures to refine the figures how do you include those in the losses, and what if the technical measures actually perform some other task other than just refining the figures?

I could go on with a myriad of other examples to show that calculating the losses is a compleate and utter waste of time and resources.

But importantly underneath it is the Achilles Heal of ICT Sec, a compleat lack of usable, testable, verifiably reliable and universaly agreed measurands. Without them we are not practicing "science" but "hocus pocus quack medicine" [1][2].

Importantly though just like "Best Practice" there is money and fame to be found in such quackery, and that works both for supporters and detractors, and often it can provide some semblance of entertainment for onlookers, so "showmanship" is often an essential ingredient as is a suitable forum...

[1] Dictionary definition of Hocus Pocus : Noun, Meaningless talk or activity, often designed to draw attention away from what is actually happening.

[2] Dictionary definitions of Quack Medicine also Quackery : A derogatory term used to describe the promotion of unproven or fraudulent medical practices often for gain by a quack [3]

[3] Dictionary definition of Quack : Noun, 1; A fraudulent or ignorant pretender to medical skill or 2; A person who pretends, professionally or publicly, to have skill, knowledge, or qualifications he or she does not possess; a charlatan.

Posted by: Clive Robinson at October 26, 2011 07:26 AM

"I've lost $90,000 plus what I still owe in phone calls and I have a debt of $15,000," Rosalie said. ...

Posted by: Did this happen? What can we do with one data point???? at October 31, 2011 10:03 AM
Post a comment

Remember personal info?

Hit preview to see your comment as it would be displayed.